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Abstract 

In Michel Talagrand gave a rigorous proof of the Parisi formula in the classical 
Sherrington-Kirkpatrick (SK) model. In this paper we build upon the methodology 
developed in and extend Talagrand's result to the class of SK type models in 
which the spins have arbitrary prior distribution on a bounded subset of the real line. 
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1 Introduction and main results. 

In [TT] Michel Talagrand invented a rigorous proof of the Parisi formula for the free energy 
in the Sherrington-Kirkpatrick model |7j. The methodology developed by Talagrand was 
based upon a deep extension of Guerra's interpolation method in [2] to coupled systems of 
spins which provided necessary control of the remainder terms in Guerra's interpolation. 
The same methodology was successfully used in [T3] to compute the free energy in the 
spherical model and in the present paper we will utilize it in the setting of a generalized 
Sherrington-Kirkpatrick model in which the prior distribution of the spins is given by an 
arbitrary probability measure with bounded support on the real line. 

Let us start by introducing all necessary notations and definitions. Consider a bounded 
set S C R and a probability measure v on the Borel a- algebra on S. Given N > 1, consider 
a product space (E N ,v N ) which will be called the space of configurations. A configuration 
a £ T, N is a vector (<Ji, . . . , cttv) of spins cr^ that take values in S. For simplicity of notations we 
will omit index N in u since it will always be clear from the context whether we consider 
measure v on S or the product measure on H N . For each N we consider a Hamiltonian 
Hn(ct) on that is a Gaussian process indexed by cr £ Y1 N . We will assume that H^(cr) is 
jointly measurable in (cr, g), where g is the generic point of the underlying probability space 
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on which the Gaussian process Hn(ct) is defined. We assume that for a certain sequence 
c(N) — > and a certain function £ : R — > K we have 



Vcr , cr £L , 

where 



^EH N (cr 1 )H N ((T 2 ) — ^(Rijt) <r(X). (Li; 



i<V 



is called £/ie overlap of the configurations cr 1 ,©" 2 . We will assume that £(x) is three times 
continuously different iable and satisfies the following conditions 

£(0) = 0, £(x) = £(-z), > if x > 0. (1.3) 

We will denote the self-overlap of cr by 



N 

i<N 



One defines the Gibbs measure Gn on E by 

dG N ((x) = —expH N (<T)dv(<r) (1.4) 

where the normalizing factor 

= / exp HKf{(r)dv{a) 

is called the partition function. This definition of the Gibbs measure also includes the case 
of models with external field because given a measurable function h(a) on E and the Gibbs 
measure defined by 



dG n(ct 

'N 



-!- exp (h n (<t) + h(aij) dv{i 

ZN V t?N ' 



we can simply make the change of measure dv'{a) ~ exp h{a)dv{a) to represent this Gibbs 
measure as (jl.4|) . The only assumption that we need to make on h(a) is that 



J exp h(a)du(a) < oo 



which holds, for example, when h(o~) is uniformly bounded. This general model includes the 
original Sherrington-Kirkpatrick (SK) model in [7] and the Ghatak-Sherrington (GS) model 
in £Q that will be considered in more detail in Section 12.11 In both cases the Hamiltonian 
H N (cr) is given by 

Hn{°) = —f=y^gij(Ji(Jj (1.5) 

ViV *r~. 



for some (3 > and i.i.d. Gaussian r.v. gij, the set £ is equal to { — 1, +1} in the SK model 
and {0, ±1, . . . , ±S"} for some integer S in the GS model and in both cases the measure v is 
uniform on S. In the case of the SK model the function h(a) is given by h(a) = ha with the 
external field parameter /i£l, and in the case of the GS model it is given by h(a) = ha 2 
with the crystal field parameter h G R. Let us define 

F N = ^ElogZ N = i-Elog I expH N {o-)dv(<r), (1.6) 

iV iV Jj;JY 

which (usually, with the factor — (3~ l which we omit for simplicity of notations) is called 
the free energy of the system (S , Gjv)- The main goal of this paper is to find the limit 
liniTv^oo Fn. It will soon become clear that the main difference of the above model from the 
classical SK model lies in the fact that in the classical model the length of any configuration 
<r G { — 1,+1} N was constant, |cr| = y/~N, which is not always true here. In general, if £ 
is not of the type {— a, +a} for some a G 1 then the length of the configuration or self- 
overlap i?x,i = \<t\ 2 /N wm become variable. As a result, in order to make the methodology 
of Guerra and Talagrand work, we will first have to compute the local free energy of the set 
of configurations with constrained self-overlap. 

Let us now describe the analogue of the Parisi formula that gives the limit of Let 
[d, D] be the smallest interval such that 

u({a:a 2 e[d,D]}) = l. (1.7) 

In other words, d < a 2 < D with probability one and a 2 can take values arbitrarily close to 
d and D with positive probability. From now on we will simply say that d < a 2 < D for all 
a G E. Let us consider u G [d, D] and a sequence (en) such that e^ > and limjv^ooejv = 0, 
and consider a sequence of sets 

= (cr G : R 1A G [u - £ N ,u + e n )\. (1.8) 
F n (u,e n ) = —ElogZ N (u,£ N ), (1.9) 

where 

Z n (u,e n ) = / expH N (a)du(a). 
Ju N 

Fn(u,En) is the free energy of the subset of configurations in (jl.8J) . We will first compute 
limjv^oo Fn(u, En) for some sequence (en) for each u G [d, D\. Consider an integer k > 1, 
numbers 

= m < m% < . . . < va^_\ < = 1 (1-10) 

and, given u G [d, D] , 

= go < qi < ■ ■ ■ < Qk < Qk+i = u. (1.11) 

We will write m = (m , . . . , m^) and q = (go, • • • , Qk+i)- Consider independent centered 
Gaussian r.v. z p for < p < k with 

Ez 2 p = e(q p+1 )-e(q p ). (1.12) 



U N 

We define 
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Given A G 1, we define the r.v. 



X k+ i = log J exp^a z p + Xa 2 ^jdu(a) (1.13) 
and, recursively for I > 0, define 



0<p<fc 



X l = — logEjexpmjXi+i, (1.14) 

where E; denotes the expectation in the r.v. (z p ) p >i. When mi = this means X; = E^X^ +1 . 
Clearly, X = X (m, q, A) is a non-random function of the parameters m, q and A. Whenever 
it does not create ambiguity we will keep this dependence implicit. Let us note that X Q also 
depends on u through q since in (ll.ll|) we have q k +i = u. Let 

V k (m, q, X,u) = -Aw + X (m, q, A) - - mi(9(qi +1 ) - 6(q t )) (1.15) 

l<l<k 

where 9(q) = q£'(q) — £(<?), and define 

V(£,u) = mf V k (m,q,\,u), (1.16) 
where the infimum is taken over all X,k,m and q. Finally, we define 

V(£)= sup V{£,u). (1.17) 

We will first prove the following. 

Theorem 1 Given u G [d,D] and a sequence (en)n>i that goes to zero slowly enough, 

lim F N (u,e N )=V(£,u). (1.18) 

Gaussian concentration of measure will imply that the limit of the global free energy 
can be computed by maximizing the local free energy. 



Theorem 2 We have 



lim F N = P(£). (1.19) 

N^oo 



Organization of the paper. In Section El we describe the replica symmetric region 
of the model and discuss the example of the Ghatak-Sherrington model. In Section El we 
introduce the construction (which we call the Parisi functional) that is used often throughout 
the paper and study some of its properties. In Section |U we prove the analogue of Guerra's 
interpolation and explain why it seems to be necessary to impose the constraint on the self- 
overlap in order to utilize the methodology of Talagrand in [TT] . Compared to the classical 
SK model where this problem does not occur, for the general model considered in this paper 
a brand new argument is required to remove the constraint on the self-overlap at the end of 
Guerra's interpolation. This constitutes a certain nontrivial large deviation problem that is 
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solved in Section In Section |H1 we show how Theorem ^ can be reduced to certain apriori 
estimates on the error terms in Guerra's interpolation. For the most part, the proof of these 
apriori estimates goes along the lines of the methodology developed by Talagrand in ^l] 
but, nonetheless, considerable effort is required to verify that the arguments and numerous 
computations in ^Tj extend to this more general model. We carry out these computations 
in Appendix A. In Section [7| we show how the global Parisi formula of Theorem El follows 
from the local Parisi formula of Theorem^ and a certain concentration of measure result. 
Finally, certain values of the parameter u in Theorem require small modifications of some 
arguments but, fortunately, these cases can be reduced to the classical model considered in 
[TT] . a work which is postponed until Appendix B. 



2 Replica symmetric region. 

In this section we will describe a relatively simple necessary and sufficient condition in terms 
of the parameters of the model which guarantees that the infimum on the right hand side 
of (jl.l6|) is achieved when k = 1. If this happens then V(£,u) will be called a local replica 
symmetric solution. In Section [B] we will explain that the cases when u = d or u = D in 
Theorem Q can be reduced to the classical SK model for which the domain of validity of the 
replica symmetric solution was described in jH] and, hence, in the rest of the paper we will 
assume that 

d<u<D. (2.1) 
If the infimum in ()1.16|) is achieved when k — 1 then 

m = (0, 1), q = (0, q, u) for some q G [0, u] (2.2) 

and A and q are the only variables in V\{m, q, A, u) which, hence, can be written as 

Vx{q,\) = -\u- ~(0(«)-%))+Elog / expH(a)du((T), (2.3) 

where 

H(a) = az Q + \a 2 + l -a\i'{u) - i'{q)). 
Let us define the (local) replica symmetric solution by 

RS(m) =inf7 ? i(g,A) (2.4) 

X,q 

and describe the criterion which guarantees that V(£, u) = RS(tt). We will prove that if (|2.1j) 
holds then the infimum on the right hand side of (|2.4jl is achieved on some A and q which, 
therefore, must satisfy the critical point conditions 

^1-^1-0 (25) 
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From now on let (g, A) be such a pair, i.e. RS(tt) = Vi(q, A). Suppose that V(£,u) = RS(tt). 
Then taking k = 2 in (jl.l6j) should not decrease the infimum on the right hand side. Let us 
take 

k = 2, m = (0, m, 1) and q = (0, g, a,u) for a G [g, «]. (2.6) 
With this choice of parameters Pfc(m, q, A, w) becomes 

$(m,a) = -Am - ^m(0(a) - 9(q)) - ^(0(u) - 9(a)) + ^ElogEiX™, (2.7) 

where 

X = [exp H'{a)dv{a) and H\a) = a(z + Zl ) + \a 2 + \a 2 {i\u) - £'(a)) (2.8) 

and where E^ = <f (g) an d = C'(«) - It should be obvious that $(1, a) = V 1 (q, A) 
for any q < a < u. The derivative of $(m, a) with respect to m at m = 1 can not be positive 
because, otherwise, by decreasing m slightly we could decrease $(m, a) that would imply 

< &(m,a) <V 1 (q,\) = RS(u). 

A simple computation gives 

d® , , 1 XX 

/(a) = — (m,a = -- (0 a - (q) )+ E— — log — - (2.9) 

JK ' dm y Jlm=1 2 V v ; yH " EiX 6 EiX v ; 

and, hence, the following condition is necessary if V(£, u) = RS(tt), 

f(a) < for all q < a < u. (2.10) 

This derivative f(a) represents what is usually called the replica symmetry breaking fluctu- 
ations. Let us note that since $(m, q) = V\(q, A) does not depend on m, we have /(g) = 0. 
Also, it is easy to check that (|2.5jl implies that /'(g) = 0. Therefore, if (|2.1()|) holds then we 
must have 

f"(q) < 0, (2.11) 

which in the SK model is called the Almeida- Thouless condition. It is believed (and numerical 

computations show) that in the classical SK model ()2.11j) implies (|2.10j) . However, we will 

give an example below where this is not the case and, therefore, condition (|2.10j) can not 

be weakened to (|2.11J) in general. We will prove that ()2.10j) is (necessary and) sufficient for 
P(£,u)=RS(u). 

Theorem 3 If a pair (A, q) satisfies \2. 5}) and \2.1Q}) then V(£, u) = Vi(q, A) and such pair 
(A, q) is unique. 

The proof of this theorem goes in parallel with the proof of Theorem ^ as will be explained 
in Section |UJ However, its proof would be immediate if we knew that the functional Vk 
defined in (II. 15(1 was convex in m. The conjecture that Vk is indeed convex in m was made 
in and in jB] where a partial result was proved. We do not give the details here but, 
shortly speaking, the convexity of Vk would imply the uniqueness of the minimum in the 
optimization problem (|1.16|) and since (|2.10|) means that the replica symmetric choice of 
parameters is a local minimum in (|1.16|h hence, it would be a global minimum. 



6 



2.1 Ghatak-Sherrington model. 

Let us consider the Ghatak-Sherrington model introduced in pQ with E = { — 1,0, +1}, the 
Hamiltonian Hn{&) defined in (|1.5j) . the measure v is the counting measure on £ and the 
external field h(o~) = ha 2 for some h G R. This choice of parameters gives 

Pi(g,A) = -Am - ^(m 2 -g 2 ) +Elog(l + 2ch(z/? v ^)exp(A + /i+ ^ 2 (w-g))), 

where z is a standard normal r.v. and we keep the dependence of V\ on u implicit. Because 
of the symmetry of the model, rather than the replica symmetric solution, one is usually 
interested in the case when V(£, u) = Vi(q, A) for q = and some A. It is easy to check that 

?El = il - 132 E ( 2 M^Vq)exp(X + h + (3\u-q)/2) y 
dq 2 2 Vl + 2ch(z/^)exp(A + /i + /3 2 (w-g)/2)/ 

and, therefore, q = always satisfies the critical point condition (|2.5j) . If V(£,u) = Vi(0, A) 
for some A then we will call V(£,u) a paramagnetic solution and denote it by PM(u). 

PM(m) = inf Vi (0, A) = inf (-Am - -/3 2 u 2 + log(l + 2e x+h+ ^ u ) 

A A V 4 

= hu + -[3 2 u 2 + inf (-X'u + log(l + 2e A ') 

4 A' \ 

12 1 

= hu + -t/3 2 u 2 + u log — h (1 — u) loj 



4 m 1 — u 

where we made the change of variable A' = A + h + /3 2 u/2. The infimum is achieved on 

1 11 

\ = -h- -8 2 u + log— -. (2.12) 

2 2(1 — u) 

Theorem El can be applied to this model to describe when the local free energy V(£,u) is 
given by the paramagnetic solution PM(u). When q = 0, the definition ()2.8|) implies that 

X = 1 + 2e x+h+ ^ {u - a) c\i(z l f3^a) = 1 + -^—e-^ a di( Zl f3^a). 

1 — u 

Using the fact that 

Ee~^ a ch(z/?v^) = 1 (2.13) 
we get E t X = 1 + u/(l - u) = 1/(1 - u) and for < a < u, 

f(a) = -\p 2 a 2 +E(1 - u + ue-^ a c\i{z(3^a)) log(l -u + ue-^ a c\i{z(3^a)). (2.14) 

Since A in ()2.12j) and q = satisfy (|2.5jl . Theorem El implies that the subset of configurations 
with constrained self-overlap R± t i ~ u will be in the paramagnetic phase, V(£,u) = PM(«), 
if and only if 

f(a) < for < a < u. (2.15) 
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Figure 1: A function f(a) for u = 0.05, /3 = 17.5 and a < u. 



It is easy to check that /"(0) = /3 2 (-l + (3 2 u 2 )/2 and, therefore, ([2~TT]l implies 

-1+/3V < or /3w < 1. 



(2.16) 



It is tempting to conjecture that (|2.1fi|) implies (|2.15jl but, unfortunately, even though it is 
expected to be true in the classical SK model it is not always true here. For example, one 
can check that for u = 0.05 and (3 = 17.5, ()2.16|) holds but ()2.15|) fails (see figure HJ). 

Next, we would like to describe the region of parameters f3 and h such that the system 
as a whole is in the paramagnetic phase in the sense that 



v(0 



sup V(£,u) 
ue[o,i] 



(2.17) 



i.e. the local free energy V(£, u) is maximized at some point u' where V(£, u') = PM(m') and, 
thus, the global free energy is given by the paramagnetic solution, V(£) = PM(a'). In figure 
|2]we show a phase diagram in the coordinates (/i/3 _1 , /3 _1 ) to compare it with pQ where the 
phase diagram was given in these coordinates. According to ifj, Regions 1 and 3 constitute 
the paramagnetic phase where 1)2.17)1 holds, and Region 2 is the spin glass phase where (j2.17j) 
fails, i.e. V(t;,u') < PM(m') for u' such that sup M "P(£, m) = V(t;,u'). We will explain how 
these regions were defined in P and argue that in Regions 1 and 3 (j2.17J) holds. We consider 
Regions 1 and 3 separately because Region 1 can be treated rigorously. 

The way figureElwas obtained in [I] is apparently as follows. The authors used the replica 
symmetric approximation which means that instead of looking at the free energy V(£) which 
by Theorem El is given by sup u "P(£, it) they considered a replica symmetric approximation 
of V(C,) given by 

supRS(u) = supinfPi(g,A). (2.18) 

u u It* 

The value provided by this approximation, in general, is not equal to the actual free energy 
V(C,) and is only an upper bound. However, this optimization problem is much easier than 
the case of the general Parisi formula in Theorem |21 since (J2.18|) depends only on three 
parameters (u, q, A). The saddle point conditions for the solution [u, q, A) of ()2.18|) are given 
by 



&Pi 



0. 



dq 



and A = 0, 



(2.19) 
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Figure 2: A phase diagram in (h/3 ,/3 1 ) coordinates. Regions 1 and 3 are a paramagnetic phase 
and, according to 1 1, Region 2 is a spin glass phase. 

since it is easy to check that maximizing over u and using dVi/dX = gives A = 0. Hence, 
(I2.19J) reduces to solving the system of two equations. It was predicted in pQ that the para- 
magnetic phase coincides with the set of parameters (/3, h) for which the infimum in (|2.18j) 
is attained at the saddle point (u, q, A) such that q = 0. This set is given by the union of 
Regions 1 and 3. On the complement, Region 2, the replica symmetric approximation of the 
free energy V(£) is given by V\(q, A) with q ^ and the authors in [I] concluded that it is, 
therefore, a spin glass phase in the sense that (j2.17|) fails. However, this conclusion in general 
requires further justification because (j2.18|) is only an approximation of the general Parisi 
formula. 

Region 1. We will define Region 1 below after we explain a simple but important 
property of this model. The fact that Region 1 is a paramagnetic phase will follow from this 
property. First, let us observe that for any fixed a the function f u (a) = /(a) is increasing in 
u, where for a moment we made the dependence on u explicit. We have 

df (a) b 2 a 2 

-^-^ = E(-l + e* a ch(zl3y/a)) log(l -u + ue-- a ch(z(3^)). 

It is easy to check that for any u G [0, 1], the function x — > (—1 + x) log(l — u + ux) is 
convex for x > and, therefore, (|2.13|) and Jensen's inequality imply that df u (a)/du > 0. 
This means that if we take u\ < u-i then 

f U2 (a) < for a < u 2 => f Ul {a) < for a < u x . (2.20) 

If for some u 2 G [0, 1] we have V(£,u 2 ) = PM(m 2 ) then $nty holds for u = u 2 . By (EM . 
()2.15|) holds for any u = U\ < u 2 and, therefore, V(£, U\) = PM(itx). This proves the following 
important property: for any (3 and h there exists u = Uq{(3) G [0, 1] such that 

V{£, u) = PM(m) if and only if u < u . (2.21) 
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Suppose that the maximum of PM(tt) is achieved on some U\ < uq. Then the system as a 
whole will be in the paramagnetic phase because 

sup V(£,u) > sup V(£,u) = sup PM(u) = sup PM(u) > sup V(£,u) (2.22) 
ug[o,i] u<uq u<uo «e[o,i] «e[o,i] 

and, therefore, 

V(0 = sup V(£,u) = sup PM(«) = PM(ui). (2.23) 

we [0,1] u<u 

Region 1 is precisely where the maximum of PM(u) is achieved on some u\ < uq and, thus, 
it is a subset of the paramagnetic phase. 

Region 3. As we mentioned above, in Region 3 the optimization problem ()2.18|) is 
solved at the saddle point («', q', A') such that q' = 0. This means that for this u', 

supRS(w) = RS(w') = inf 7>i(g,A) = infPi(0,A) = PM(«'). (2.24) 

By itself this fact does allow us to conclude that we are in the paramagnetic phase, but 
apparently in this particular model when this happens we also have u' < Uq where u$ was 
defined in (j2.21j) and we do not see how to prove this using calculus. Should this numerical 
observation reflect the true situation, as seems likely, then (|2.24j) would imply that 

PM(w') =V(^u) < supP(£,u) < supRS(n) = PM(u') 

u u 

and we would again be in the paramagnetic phase. 

Region 2. The fact that the infimum in ()2.18j) is attained at the saddle point {v! ', q', A') 
with q' 7^ implies that u' > Uq and 

sup PM(m) < RS(w'). (2.25) 

However, since RS(w) is only an upper bound on V(£, u), in general, (|2.25|) does not exclude 
the possibility that 

V(£,u) < sup PM(w) for all u > u 

u<uo 

which would imply ()2.17|) . At this moment we do not see how to prove that Region 2 is a 
spin glass phase except by checking directly that (|2.17|) fails. 

3 Parisi functional. 

We will often consider iterative constructions similar to ()1.13|) and (|1.14jl . so it will be 
convenient to define an operator that implements this recursion. In each case we will only 
need to specify the parameters of the operator. We will call this operator the Parisi functional. 
Given k > 1, consider a vector 

m = (m , . . . ,m k ) (3.1) 
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such that all coordinates m ; > 0. Let z = (zi)o<i<k be a collection of independent random 
vectors. 

Suppose that we are given a random variable F that is a function of z, F = F(z). Then 
we let Fk+i = F and for < I < k define iteratively 

F, = — bgE,expm,F l+1 , (3.2) 
mi 

where E; denotes the expectation in (zj)j>;. When mi = 0, this means Fi = EjFj +1 . 
Definition. We define the Parisi functional by 

V(m)F = F Q . (3.3) 

With these notations the definition of X given by (jl.lOj) - (jl.l4j) can be written as 

X = V(m)X k+1 . (3.4) 

Let us describe several immediate properties of the Parisi functional that will be often used 
throughout the paper. It is obvious by induction in ()3.2j) that for any constant eel, 

V(m)(c + F) =c + V(m)F. (3.5) 

Similarly, if z^ = (z0o<p<k are independent for j = 1, 2 and F^ = F^(z^) then 

V(m)(F 1 + F 2 ) =V(m)F 1 + V(m)F 2 . (3.6) 

If F < F' then 

V{m)F < V(m)F'. (3.7) 

The next property plays an important role in Talagrand's interpolation for two copies of the 
system. Suppose that we have two random variables 



F' = F(J24} for j = 1,2 (3.1 



p<k 

such that 

z l = zf for / < r and z\ and zf are independent copies for I > r. (3.9) 
Let us define n = (n p ) p <k such that 

Tfli 

ni = — for I < r and ni = mi for I > r. (3.10) 

Lemma 1 Given $2$, / TO) and fiTM let F = F 1 + F 2 and define Fj by UTty and F l by 
\3. with m replaced by n. Then, 

F t = Fl + if forl>r and F l = IF} = 2F 2 for I < r. (3.11) 

In particular, 

V(n)(F l + F 2 ) = 2V{m)F\ (3.12) 



11 



Proof. The proof follows by induction in (|3.2|) . For / > r, using independence of z\ and zf 
we get 

F t = - logE, exp mF l+1 = — logE, exp m,(i? +1 + if +1 ) = if + if. 

mi 

For / = r all independent copies already have been averaged and since z\ = zf for I < r, 
if = if and F r = IF}. . Finally, by induction for I < r we get 

Ft = — logEi exp mF l+1 = — logE, exp ^2if +1 = 2if 
ni mi 2 

which for I = proves (J3.12)) . 

□ 

The next property concerns the computation of the derivatives of V(m)F. With the 
notations of (|3.1|) and ([3.2)1 . let us define 

W, = expm,(F, +1 -.F 1 ). (3.13) 

Note that by definition of Fi in f)3.2|l we have E{Wi = 1. Also, since i 7 }, F i+ i and W 7 ; do not 
depend on Z{ for z > I + 1, we can write 

EjWjWI+i = E,E I+a W,W, + i = E,W,E, + iW, + i = 1. 

Repeating the same argument, 

E { W,...W fc = l. (3.14) 

This fact will be used often below. 
Lemma 2 For a generic variable x, 

W £. (3.15) 

ox ox 

Proof. By (|3.2|) . expm^F^ = E;expm^F i+1 and, therefore, 

miexpmiFi—- = m^Ej expmzF m — — . 

ax ax 

Since i 7 ) does not depend on Zi for i > I, 

ox ox 

where W\ was defined in f!3.13|) . Applying the same equation to gives 

dFi 9Fi +2 „.„, dF+2 

— = EjWiEj+iWi+i— — = EiWiW l+1 — — 
ox ox ox 

since Fi,Fi + i and, therefore, Wi do not depend on (z^ for z > Z + 1. Repeating the same 
argument inductively we get 

r)F r)F 

^ = E l Wi...W k — , (3.16) 

ox ox 
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which for I = implies (J3.15|) . 

□ 

We will often assume that the coordinates of m are arranged in a nondecreasing order, 

m < . . . < m k . (3-17) 

The following lemma provides a useful control of the expressions of the type f)3. 15j) that 
appear as the derivatives of the Parisi functional. 



Lemma 3 Suppose \3.11 ) holds. Let f = f(z) and F = F(z) and let W\ be defined by 
\3.13\) . Suppose that f < F and let m = m r be the first nonzero element in \3. 17]) . Then 

E log E r W r . . .W k exp m k (f — F) < m(P(m)f - V{m)F). (3.18) 

Proof. Let us define U = expm,t(/ — F). Using (J3.13)) . we can write 

W k U = expm k (f - F k ) 

and since F k does not depend on z k , this implies that 

E k W k U = exp(-m k F k )E k expm k f = expm k (f k - F k ). 

We will proceed by induction to show that for r < I < k, 

EiWi-.-WkUKexprmifi-Fi). (3.19) 

As in (J3.7J) . f < F implies that fi < Fi and since m/_i < mi, (I3.19|) implies that 

EiWi . . . W k U < expm^ifi - F t ). 

Multiplying both sides by Wi-i gives 

EiWi-iWi ...W k U< expmi-i(fi - 

since Wi does not depend on z% for i > I. Taking the expectation E^_i and using that 
does not depend on z% for % > I — 1 we can write 

Ei^Wi-iWi . . . W k U < exp(-m i _iF;_i)E i _iexpmi_i/ / = expmn(/i_i - 

This finishes the proof of the induction step. For I = r, (|3.19j) implies that 

\ogE r W r ...W k U <m(f r - F r ). (3.20) 

Since for I < r, mi = 0, ()3.2j) implies that 

V(m)f = f = E f r and V{m)F = F = E F r 

and, therefore, taking the expectation of both sides of (J3.20|) proves (|3.18|) . 
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4 Guerra's interpolation. 

The first step of the proof of Theorem ^ is the analogue of Guerra's interpolation method 
in |2j. For 1 < i < N, we consider independent copies (zj iP )o<p<fc of the sequence (z p )o< P <fc 
defined in (jl.l2|) that are also independent of the randomness of the Hamiltonian H^(cr). 
Consider m = (m p )o< p <fc as in (jl.lOj) . We denote by E z the expectation in the r.v. (z itP )i<N,p>i- 
Consider the Hamiltonian 

i<N 0<p<k 

Given Un defined in fll.8|) . let 

F = log / expH t {a-)du((T) (4.2) 
Ju N 

and let 

<p N (t) = ^EV(m)F, (4.3) 

where E denotes the expectation in all random variables including the randomness of the 
Hamiltonian Hn{ct)- 

For a function h : Y, N — > R, let (h) t denote its average with respect to the Gibbs measure 
with Hamiltonian H t (cr) in (|4.1j) on the set Un, i-e. 



(h) t expF= / h{a-)expH t {cr)du((T). (4.4) 

JUn 

1)3.14)1 implies that the functional 

h^E l (W l ...W k (h) t ) (4.5) 

is a probability 7/ on Un- We denote by jf 2 its product on Un x Un, and for a function 

h : Un X Z7jv — >■ K we set 

^(A)=E(^ 1 ...^,_i 7 f 2 (/i)). (4.6) 

The following Gaussian integration by parts will be commonly used below. If g is a Gaussian 
random variable then for a function F : R — >• R of moderate growth we have (A. 40 in jHj), 

EgF(g)=Eg 2 EF'(g). (4.7) 

This can be generalized as follows. If g — (gi, . . . , g n ) is a jointly Gaussian family of random 
variables then for a function F : R n — > R of moderate growth we have (see for example, A. 41 
in 0), 

dF 

E 9l F(g) = ^E(^)E— (<?). (4.8) 

We will need a similar statement for functionals of not necessarily finite Gaussian families, 
for example, for a random process Hn{ct) indexed by a in a possibly infinite set S . The 
following is a simple consequence of (|4.7j) . 
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Lemma 4 Let g = (g(p)) p eu be a Gaussian process indexed by U C R n and let F(g) be a 
differentiable functional on M. u . Given a G U, we have 

Eg(a)F(g) = E^[Eg(a)g(p)] (4.9) 

- the expectation of the variational derivative of F in the direction h(p) = Eg(cr)g(p). 
Proof. Consider a process g' = (g'(p)) P eu denned by 

Eg(p)g(<r) 



g'(p) = g(p) -g{<7) 



Eg(tr)< 



which is, obviously, independent of the r.v. g(cr). If we fix g' and denote by E' the expectation 
with respect to g(cr) then, using ()4.7|) with g = g(cr) gives 

E'g(a)F(g) = E'g(cr)F (g\p) + g{a) ) = E' 5 -^[Eg(cr)g(p)]. 

Taking the expectation in g' proves ()4.9j) . 

□ 

We are ready to prove the main result of this section. The proof will clarify why we 
first compute the free energy of the set of configurations with constrained self-overlap G 

[u — e N , u + ejv]- 



Theorem 4 (Guerra's interpolation). Fort G [0,1] we have 
<P' N (t) = ~J2 mi(9(qi + i) - 9( qi )) 

l<l<k 

-\Y,^ mi - m '-i)^ (£( R W) ~ R ^'(Qi) + 0(qi)) + K, (4.10) 

l<l<k 

where \1Z\ < c(N) + Le N . 

Proof. The proof of this theorem repeats the proof of the main result in [2j (see also Theorem 
2.1 in JT]) with some necessary modifications. We will give the detailed proof in order to 
demonstrate how Lemma E| replaces (|4.8|) and to show that the constraint q k+1 = u in (jl.llj) . 
in some sense, matches the constraint on the self overlap R^i G [u — e^,u + £jv] in the 
definition of local free energy (jl.8j) . (jl.9j) . In particular, we will see in (14.2 1|) below that 
such choice of parameters allows us to get rid of a certain term in the derivative tp' N {t) that, 
otherwise, would be problematic to control. First of all, f)3.15j) implies that 

1 dF 
(p ' N (t) = -EW 1 ...W k —. 
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Using the fact that = 1, one can write 

W x ...W k = exp ^ rni{F l+l -Ft) 

l<l<k 

= exp(F + ^ (m^x-m^F^ = TexpF, (4.11) 



KKk 



where T = T\ . . . and Ti = exp(m;_i — m{)Fi. Using ()4.2j) we can write 

1 dF 
<p' N (t) = -ETexpF— = I-II, 



where 

'u N 



l=— ]-= ! ETH N (tr)expH t (tr)dv(tr). (4.12) 
2J\y/t Ju N 



and 

II 



)== I ET^^^z^expi^o")^)- (4.13) 

i<N p<k 

To compute I, we will use (|4.9jl for the family gt = (Hjf(p)) p! zu N and we will think of each 
factor Ti in T as the functional of g. Let us denote 



C(<T,p) = ±EH N (<T)H N (p). (4.14) 



Then (jOJ) and (jUH) imply 



1 = ^ / ET«Wc(-, CT )rfK-) (4.15) 
2Vi Ju N oH N (tr) 



Kk 

First of all, 



+ AE/ ETexp^^i^fC^p)]^) 



dexpH t (cr) r 
dH N (a) =^exp^( g ), 



and, therefore, the first line in (|4.15J) can be written as 

i / ETexpfT t («r)C(ff,ff)dKff) = iEW 1 ...W' fc (C(<r,<r)> t . (4.16) 
Using the definition of T} we can write 

i^[C(<r, p)] = (m,_i - m,)^[C(<r, p)] = K-i - m,)Wi . . . W^[C(<x, p)], 

where we used f)3.16j) . We have 

SF f 

— [C(<x,p)] = v / texp(-F) / ((cr,p)exvH t (p)du(p) = p))' 
°9 Ju N 
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where (■)' t denotes the Gibbs average with respect to p for a fixed cr. Therefore, for a fixed 
a we get 

1 ST 

H 5^ [C(<T ' P)] = ^ m - 1 ~ m ^ Wl ■ ■ ■ W k{C(^, P))' t = Vtirm-! - rrHMda, p)), 
where 7; was defined in (14. 5|) . Hence, the second line in (|4.15|) is equal to 

iV)(m,_ 1 -m I )EW 1 ...W fc ex P (-F) / 7i(C(<r, p)) exp# t («r)tM<r) (4.17) 



Kfe 



= \ E( m '-i - m ^) EW/ i • • • w i-*y? 2 (c(°-- p)) = ^ E^- 1 - m iMC(<r, p))> 

where \ii was defined in f!4.6|) . Combining ()4. 16|) and ()4.17|) we get 

I = -EWk . . . W k {((<r, °)) t + \ ^(m H - rudder, p)). (4.18) 

Kfc 

By (jl.lj) . for any cr 1 , cr 2 £ we have 

|C(tr\crV£(^i,2)|<c(iV) 
and, therefore, ()4.18|) implies that 

I = ^EW 1 . . . W k (Z{R ltl )) t + i - m,)^(e(i2i, 2 )) + ft, (4-19) 

where |7£| < c(N). The computation of II is very similar, one only needs to note that F\ does 
not depend on z^ p for I < p. We have 

II = -C'te+OEW! . . . W k (R ltl ) t + \ J2( m i-i ~ m^'iqMR^). (4.20) 

l<k 

Combining (J4.19|) and (|4.2U|) and rearranging terms, it is easy to see that 

<p' N {t) = l -W, . . . W k (Z(R 1;1 ) - i?i,i£Wi) + 0(q k+1 )) t - ^mi^fe) - 9{ qi )) 

l<k 

- \ E ^ " m '-^ ft(^. 2 ) - ^C'fe) + %)) + ft- (4-21) 

i<z<fc 

It now suffices to notice that (-) f is restricted to the set Un and q k +i = u by (jl.llj) . which 
implies that 

< £(Ri,i) - Ri,i£'(u) + 9(u) < L\R hl -u\< Le N . 

Since EWi . . . = 1, this finishes the proof of Theorem 0] It is to control the first term on 
the right hand side of (|4.21j) that we impose the constraint on self-overlap. In the classical 
Sherrington-Kirkpatrick model this problem did not occur because was always 1. 

□ 
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5 Removing the constraint on the self-overlap. 

The main goal of this section is to compute limjv^oo <£>iv(0). The convexity of £ implies that 
£(a) — a£,'(b) + 6(b) > for any a, 6 G R and, therefore, (|4.1()jl implies 

<S N (t) <-\j2 m i(8(<H+i) - Hit)) + K 

l<l<k 

and, hence, 

F N (u, e N ) = <p N (l) < <p N (0) -\Y, m i(0(<li+i) ~ %0) + ^- (5-1) 

i<«<fe 

In the SK model ^at(O) was very easy to compute and, in fact, it was independent of N 
because at the end of Guerra's interpolation the spins became decoupled. The situation is 
different here because of the constraint (jl.8j) . Let us recall that for t — 0, 



(0) = ^V(m)F where F = log / exp Oj ( ^ z it ^jdv(cr). 

Ju N i< N o< p <k 



Removing the constraint Un constitutes a large deviation problem that will be addressed in 
Theorem below. First of all, let us give an easy upper bound on ^(0). Since by (jl.7|) the 
self-overlap G [d, D], for cr G Un we have Ri^ G [d, D] fl [it — e^, u + £jv] an d, therefore, 

-Ai2i,i < -Aitjv(A), (5.2) 

where 

^iv(A) = max(<i, w — £#) f° r A > and un(\) = mm(D, u + e^) for A < 0. (5.3) 
Using this, we can bound F as follows, 

F = log / exp^^o-j ^ z p ^du((T) 



< —NXum(X) + log / exp f Oj + A ^ cr, 2 ) rii/(tr) 

i<iV 0<p<fc i<N 

(A) + log / exp °i Yl z p>* + X Y a i) dv ^) 



< -NXu N 
= -NXu N (X) + J2 x k+i, 



i<N 0<p<k i<N 



i<N 



where 



X k+lji = log J ( a Yl + X(y2 ) dv ( Cr ) ( 5 - 4 ) 



0<p<fc 
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are independent copies of Xk+i defined in (J1.13J) . Using (|3.5j) and ()3.7|) . 

<p N (Q) = ^V(m)F < -Xu N (X) + V(m)X k+l = -Xu N {X) + X . 
The arbitrary choice of A here implies that 

<p N (0) < M(-Xu N (X) + X ). (5.5) 

A 

Combining (|5.1j) and (|5.5|) we get 

F N (u,e N )<inf(-Xu N (X)+X (m,q,X)-~ ^ mi (% w ) -%,))) (5.6) 

l<l<k 

where the infimum is over all choices of parameters k,m,q and A. The bound (|5.6|) is the 
analogue of Guerra's replica symmetry breaking bound in 0. If instead of mat (A) we had u 
in (15.6)1 then the infimum would be equal to V(£,,u) which would prove the upper bound in 
Theorem ^ We will now show that for 

d<u<D (5.7) 
this infimum is not changed much by replacing un{X) with u. We will need the following. 

Lemma 5 There exists a function a(A) such that for any k, m, q, 

a(A)A < X (m, q, A) (5.8) 

and such that 

lim a(A) = D and lim a(A) = d. (5-9) 

A^+oo A — > — oo 

Proof. Indeed, if in the recursive construction (jl.l4|l one takes all mi = then Holder's 
inequality yields that for any sequence m 

Elog / exp^cr 

z p + Xa 2 ^jdv{a) < X (m, q, A) (5.10) 

0<p<k 

Since the function 

X — > log y exp^crx + Acr 2 j(iz/(cr) 
is convex by Holder's inequality, (|5.10|) and Jensen's inequality imply that 

log / exp Xa 2 du(o-) < X Q (m, q, A). (5-11) 
is 

It is clear that ()1.7)) implies that the left hand side of ()5.11|) is asymptotically equivalent to 
DX for A — > +oo and to dX for A — > — oo and this proves Lemma El 

□ 

Next we will show that the estimate (|5.8|1 and equation ()5.6|) imply the upper bound in 
Theorem ^ 
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Lemma 6 For d < u < D we have 

limsupF7v(u,ejv) < V(£,u). (5.12) 

Proof. Lemma El implies that 

-Xu N (X) + X (m, q, A) > A(o(A) - u N (X)). (5.13) 

For d < u < D, the definition ()5.3j) implies that for any A, limjv^oo mat(A) = w. Combining 
this with ()5.9|) yields that for iV large enough the right hand side of ()5.13j) goes to infinity as 
A — > ±oo and, thus, the infimum in (J5.(i)) is achieved for |A| < A where A is a large enough 
constant independent of k, m, q. This means that in ()5.6|) restricting minimization over A to 
the set {|A| < A} does not change the infimum and, therefore, 

F N (u,e N ) < mf(-Xu + X (m,q,X) - - ^ - 0(«))) + A\u N (\) - u\ +TZ, 

\<l<k 

= V(£,u) + A\u N (X)-u\+ll. 
and this finishes the proof. 

□ 

In the rest of the section we will show that the bound in ()5.5|) is exact in the limit. 
Theorem 5 For any d < u < D, if the sequence En goes to zero slowly enough then 

lim (Pn(0) = ipo(m, q) '■= inf(— Am + X (m, q, A)). (5.14) 

N^oo A 

Proof. Given a measurable set A C [d, D] and A G R, we define 

„ N N 

F(AA) = log / exp^or, J2 z i,P + ^J2^) du ^ ( 5 - 15 ) 

J{Ri,ieA} i=1 < p < fc i=1 

and 

<S>(A,X) = ±V(m)F(A,X). (5.16) 
When A = [d, D], the set {R ltl E A} = and, therefore, 

F([d,D],\) = J2 X ^ 

i<N 

where X k+lji were defined in (J5.4J) . Using ()3.6|) . 

®([d,D],\) = V(m)X k+1 = X (X). (5.17) 

Here we made the dependence of Xq on A explicit while keeping the dependence on other 
parameters implicit. For simplicity of notations we will write 

F(X) := F{[d, D],X) and $(A) := D],X) = X (A). 
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In these notations Theorem El states that there exists a sequence En — > such that 

lim $(U N , 0) = inf (-Am + $(A)) = inf(-Aw + X (X)). (5.18) 

N— >oo A A 

Let us define A (it) to be the point where the infimum in (|5.18jl is achieved, i.e. 

X (X(u)) - X(u)u = inf(X (A) - Am). (5.19) 

The infimum is, indeed, achieved because of the following argument. A function — Xu + Xq(X) 
is convex in A by Holder's inequality. Lemma El implies that Xq(X) > a(X)X for some function 
a(A) such that liniA^+oo = D and liniA^-oo = d. Hence, for d < u < D, the convex function 
—Am + X (A) — > +oo as A — > ±oo and, therefore, it has a unique minimum. The critical 
point condition for X(u) is 

flf (*(«)) = u - ( 5 ' 2 °) 

Consider a fixed small enough e > such that d < u — e and u + e < D and let U £ = 
[u — e,u + e]. Let us analyze $(U S , \(u)). Consider a set V equal to either [d, u — e) or 
[u + £, D] and note that 

[d, D] = U £ U [d, u-e)U[u + e,D]. 

We will start by proving an upper bound on $(V, A (it)). We will only consider the case of 
V = [u + s, D] since the case V = [d,u — e] can be treated similarly. Since 

— 7-Ri.i < + £) fo r R\,i £ V and 7 > 0, 

we get that for 7 > 0, 

F(V, X(u)) < -Nj(u + e) + F(X(u) + 7). 
Using (|H.5jl and (|H.7j) we get 

A(m)) < C/( 7 ) := - 7 (u + e) + $(A(m) + 7) = - 7 (u + e) + X (A(m) + 7). (5.21) 
Setting 7 = gives 

tf(0) = $(A(w))=X (A(w)). 
Next, the right derivative of ?7( 7 ) at zero is 



dU 

9 7 



(9X 

= -(u + £) + — ^t— (A(it)) = -(« + e) + w = -e 
7=0+ aA 



using (|5.2()jl . Finally, it follows from a tedious but straightforward computation which we 
will omit here that 

d 2 U 



d"f 2 



< L 
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for some constant L that depends only on the parameters of the model £ and v. Therefore, 
minimizing over 7 > in the right hand side of (j5.21|) gives 
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$(V,A(u))<$(A(«))--. (5.22) 
The same bound holds for V — [d, u — e]. For / > 1, let 

Wi = expm,(F /+ i(A(«)) - F,(A(u))) 

be defined by KTl with F = F(X(u)). Given a set A C [d,D] let G A)) denote the 

Gibbs average defined by 

(I(i? M G A)) = exp(F(A, A(u)) - F(A(u))). 

The following Proposition is the crucial step in the proof of Theorem El This type of compu- 
tation was invented by Talagrand in ^1] in order to control the remainder terms in Guerra's 
interpolation and we will use this argument with the same purpose later in the paper as well. 

Proposition 1 Assume that for A C [d, D] and for some e' > we have 

$(AA(«))<$(A(«))-e / . (5.23) 

Then, 

EWi... W k G A)) <Lexp(-^), (5.24) 
where L does not depend on N. 

Proof. The proof is based on the property of the Parisi functional described in Lemma El 
For simplicity of notations let us assume that m\ > 0. The case when several elements of 
the sequence m are zeroes can be handled in exactly the same way. Let 

/ = F(A, A(u)) and F = F(X(u)) 

so that the condition / < F of Lemma El is satisfied. Lemma El and ()5.23|) imply that 

ElogEiWi.--Wfcexp(/-F) < m x {V{m)f - V(m)F) 

= Nmi($(A, X(u)) - $(A(u))) < -Nm x e' . 

Let us consider a function 

(f>(Z) = logEi^i . . . W k exp(f - F) where Z = (z i>0 )i< N . (5.25) 
We will show that <p(Z) is a Lipschitz function of Z : 

\(f)(Z) - <f)(Z')\ <LVN\Z-Z'\. (5.26) 
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First of all, 

1 /2 

x>(^o - <o) < - si (x>f) < v^i^ - z'\, 

i<N i<N 

using p.7p . Definition ()5.15|) implies that for any set A and any A, 

\F(A, \){Z) - F(A, X)(Z')\ < VND\Z - Z'\, 

where we made the dependence of F(A, A) on Z explicit. In particular, this holds for / and 
F. It is also clear from the properties (|3.5j) and (J3.7)) that iteration (J3.2)) in the definition of 
the Parisi functional preserves Lipschitz condition and, therefore, 



\Fi{Z) - Fi{Z')\ < VND\Z - Z'\ for I < k. 
Using (|4.11jl we can rewrite <f>(Z) as 

4>{Z) = logE ie xp(/(Z) + " m i)F(Z) 

l<l<k 

and (|5.26|) is now obvious. Gaussian concentration of measure (see, for example, Theorem 
2.2.4 in [8 ) implies that for any t > 0, 

P(|0(Z) - E0| > Nt) < 2exp(-Nt 2 /L). (5.27) 

In particular, with probability at least 1 — 2exp(— N/L), 

and, therefore, with probability at least 1 — 2exp(—N/L), 

Ei Wi ...W k exp(/ — F) < exp(-N/L). (5.28) 

Since / < F, 

ExWi ...W k exp(/ - F) < EtWt ...W k <\ 
using (j3.14j) which together with ()5.28|) implies that 

EWk ...W k exp(/ -F)<L exp(-N/L). 



Corollary 1 For any e > we have 

&(U e , AH) > $(A(u)) -5 N = X (X(u)) - 8 N , (5.29) 

where liniTv^oo 5n = 0. 
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Proof. ()5.22j) and ()5.24|) imply that for V equal to either [d, u — e] or [u + s, D] we have 

EWi . . . W k (I(R ltl G V)) < Lexp(-^) 

and, therefore, 

EWi...W fc (I(i2i,i £?7 e )> <Lexp(-^). (5.30) 
Suppose that ()5.29|) is not true which means that for some positive e' > we have 

<f>(U £ ,X(u))<<S>(X(u))-e', 
for some arbitrarily large N. Then again (|5.22j) and (|5.24|) would imply that 

EWi . . . Wfc^fli,! e C/ £ )} < Lexp(--J. 
Combining with ()5.3()|) we would get 



1 = EW 1 ...W k < Lexp(- 



iV 



L 

and we arrive at contradiction. 

□ 

In order to bound <&(U £ , \(u)) in terms of <&(U £ , 0), we can write 

r N 

F{U £ ,\(u)) = log/ exp(YVi J2 Zi0 + N\{u)R ltl )dv{tr) 

J{Ri,ieu e } \ =1 < p < fc y 

< JVA(«)u + iV|A(u)|£r + F(C/ e ,0). 

Using and (jSHj), 

$(tf e ,A(u)) < $(l7 B ,0) + A(u)u+|A(u)|e 

and, therefore, 

$(C4,0) > -X(u)u + X (X{u)) - \X{u)\e-5 N . (5.31) 
This implies that for any e > 0, 

liminf $(f4,0) > -X(u)u + X (X(u))-Le. 

N—>oo 

Clearly, this means that one can choose a sequence En — > such that for Un = U EN , 

liminf $(U N ,Q) > -X(u)u + X (X{u)). 

N—*oo 

Since a similar upper bound is obvious this finishes the proof of Theorem 03 
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6 Reduction of the main results to apriori estimates. 

Now that we understood what happens at the end of Guerra's interpolation we will turn to 
analyzing the remainder terms in the second line of (|4.1()|) and, in particular, the functional 
\i T defined in (|4.6j) . First of all, for a function h on Un x Un the definition of fi r (h) can be 
written equivalently as follows. Let (z p ) for j — 1, 2 be two copies of the random vector (z p ) 
defined in (I1.12J1 such that 

z p = for p < r and z , z 2 are independent for p > r. (6.1) 

Let zi = (z{ p )i<N,p<k where (z 3 ip ) p <k are independent copies of the vector (zp 1 ) for i < N. Let 
be defined by (|4.2jl in terms of 2 J and let Wf be defined by ({SHE} in terms of FK Let us 
consider the Hamiltonian 

H t (a\a 2 ) = {VtH N (^) + VT^t^4( £ 4p)) ( 6 ' 2 ) 

.7=1,2 i<V 0<p<fc 

and define the Gibbs average (h) of h by 

(/*) exp^ 1 + F 2 ) = j h(a\ a- 2 ) exp H t (cr\ a 2 ) dp (a 1 ) dp (a 2 ). 

J U N xU N 

Then, the definition ()4.6|) is equivalent to 

ii r (h) = WWl . . . Wl_,WlW 2 . . . W}W*{h). (6.3) 

We simply decoupled the measure jf 2 by using independent copies of z ijP for p > r. Using 
Lemma n we can rewrite this in a more compact way. Let F = F l + F 2 and define n as in 
dSHDJ), i-e. 



777 

Tip = for p < r and n p = m p for p > r. (6.4) 
Lemma H then implies that for / > r 

WlW 2 = expm,(if +1 ~ F i) expm/(i? +1 - F 2 ) = expn,(F, +1 - F,) 

and for Z < r 

W, 1 = exp mi{Fl +1 - Fl) = exp m(F l+1 - F t ). 

Therefore, if we define 

Wi = expm(F l+1 - Fi), 

()6.3|) becomes 

fi r (h) =EW 1 ...W k (h). (6.5) 
In particular, if h = 1(A) is an indicator of a measurable subset A C Un x t/jv an d 



F(A)=log / expH t (a\cr 2 )du((T 1 )du(a 2 



(6.6) 
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then, since 

F = F 1 + F 2 = \og! expH t {tr 1 ,tT i )dv(<T 1 )dv(tr i ) > (6.7) 

JUnxUn 

we get 

t i r (I(A))=EW 1 ...W k eMf-F). 
Lemma El provided the methodology to control this expression. 

Lemma 7 Let F(A) and F be defined by 16. 6]) and \6. 7\ ). If for some e > we have 

^EV{n)F{A) < 2ip N {t) - e (6.8) 

then for some constant K independent of N and the set A, 

fi r (I (A)) < Kexp(-N/K). (6.9) 
Proof. Using (j3.12j) and ()4.3|h we can write 

^EV(n)F = ^EV{m)F l = 2cp N (t), 

so that ()6.8j) can be written as 

KP(n)F(A) - EV(n)F < -Ne, (6.10) 

which is the type of condition used in Lemma El The main idea was already explained in 
detail in the proof of Proposition ^. 51 However, the function that was defined in ()5.25|) was 
a function of the finite Gaussian vector Z in M. N with independent coordinates which allowed 
us to use the classical Gaussian concentration of measure inequality in ([5.27)1 . Now, however, 
both F(A) and F depend on the entire Gaussian process Hn(ct) indexed by cr e T, N and 
the only information that we specified about this process was the covariance operator in 
(|l.ip . Still, using the specific definition of F(A) and F and (jl.l|) one can prove the same 
concentration inequality as (|5.27|) but it would require a tedious computation repeating the 
proof of (|5.27|) in |8. a . We will actually carry out this computation in a relatively easier 
situation, below Lemma |H1 so the idea will be clear and we will omit this computation here. 
The proof becomes more transparent when the Hamiltonian is expressed explicitly in terms 
of an i.i.d. Gaussian sequence. For example, one often considers a Hamiltonian of the type 

Hn{(t) = N (p-i)/2 9n,.,i P °n ■ ■ ■ o- ip , (6.11) 

P>1 h,—,ip 

where g = (g^ % ) is a sequence of standard Gaussian random variables independent for all 
p > 1 and all (ii, . . . , i p ). In this case, 

^^((T 1 )^^ 2 ) = £(i2i, 2 ) where £(x) = ^a 2 x p . 

p>i 
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The sequence (a p ) p >i should be such that £(-Ri,2) is well defined for all cr 1 , cr 2 and, comparing 
with ()1.7|) this means that £(D) < oo. It is easy to check that for two sequences g and g' we 
have 

\H N (<r)(g) - H N (<r)(g')\ < ^WD)\g - g'\ 

and since inequality (|5.27|) is dimension independent, it applies to a sequence g and the rest 
of the proof repeats the proof of Proposition 13.51 

□ 

In Lemma |H1 we explained why is an upper bound on local free energy and the 

main reason was that the remainder terms in Guerra's interpolation were nonnegative. In 
order to show that this bound is exact in the limit we must show that these remainder terms 
are small along the interpolation for some choices of the parameters k, m and q and that 
(fisr(t) can be approximated by 

i;{t) = tp (m, q) - ~ Yl ™K%m) ~ 0(qi)), (6.12) 

l<l<k 

where ip (m,q) was defined in (j5.14j) . It is also clear that these parameters should approxi- 
mate the infimum in the definition ()1.16j) of V(^,u). 

Definition. We will call a vector (k, m, q, A) an e-minimizer if 

P fc (m, c/, A, u) <V(i,u) + E and (m, q, A) is the minimizer of ()1.15j) . (6.13) 

For any v G [-D, D], let us define a set 

A(v) = {|i? 1)2 - v\ < ^} f](U N x U N ). (6.14) 

The following apriori estimate will allow us to control the remainder terms in Guerra's 
interpolation. 

Theorem 6 For any to < 1 there exists e > that depends on to,C,,u,u only such that if 
\6. holds then for t <to and for large enough N, 

±W>{n)F{A{v)) < 2i;(t) - + TZ, (6.15) 

where K is a constant independent of N,t and v and \H\ < a N for a sequence (a^v) indepen- 
dent oft and v and limjy^oo a N = 0. 

In the case of the replica symmetric region of Theorem El the condition on e-minimizer 
is replaced by the condition of stability to replica symmetry breaking fluctuations defined in 

tjnnj). 

Theorem 7 Suppose that all functions are defined in terms of parameters in i2.fy) and that 
and \2.1U\) hold. Then for any to < 1 and for any t < t Q , for large enough N, 

±KP(n)F{A{v)) < 2m - + K, (6.16) 

where K is a constant independent of N,t and v and \1Z\ < for a sequence (on) indepen- 
dent oft and v and limjv^oo = 0. 
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The proof of these apriori estimates will be postponed until Appendix A. First, let us 
show how they imply Theorems ^ and EJ 

Proof of Theorem^ Given to < 1 let us take e > as in TheoremEland let (k, m, q, A) 
be an e-minimizer defined by (jfi.lHj) . Clearly, in this case, 

if>(l)=V k (m,q,\,u) and |V(1) - V(£, u)\ < e. (6.17) 

Let us take K as in ()6.15|) and for E\ > define a set 

V = {v G [-D, D] : (v — q r ) 2 > 2K(iP(t) - (p N (t)) + 2K £l }. 

For any v G V, ()6.15|) implies that 

±-EV(n)F(A(v)) < 2^(t) - ( ^ ~ ^ +U< 2ip N {t) - e x 

iv K 

for large enough N. Everywhere below let L denote a constant that might depend on e\ and 
K denote a constant independent of E\. Applying Lemma d we get 

Hr(I(A(v))) < Lexp(-N/L) (6.18) 

and the constant L here does not depend on v. Let us consider a set 

A = {(R 1>2 - q r f > 2K(i>(t) - <p N (t)) + 2K £l } f](U N x U N ). 

We can choose the points t>i, . . . ,vm G V with M < KN such that A C [j i<M A{vi) and 
f!6.18|) implies that 

Vr(I(A)) < LN exp(—N/L) < L exp(— iV/L). (6.19) 
Using the definition of if) in (I6.12j) and (|4.1()j) . 

l<l<k 

Since the second derivative of £ is bounded on [-D, D], 

Z(Ri,2) ~ Ri,2?(qi) + 0(Qi) < K(Ri, 2 - qi f 
and because on the complement A c of A we have 

(#1,2 - <?r) 2 < 2A-(^(t) - <^v(t)) + 2Ks u 
(I6.19|) for r = I implies that 

M£(#i,2) - i*i, 2 £'(ft) + %*)) < ^((#1,2 - qif) 

< K^(t) - <p N (t)) + e x + tn(I(A))) < K< ^(f) - MQ) + K£ i + Lexp(-N/L). 
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()6.20j) now implies that 

(i/)(t) - <p N (t))' < K(i/)(t) - <p N (t)) + Ke x + Lexp(-N/L). 

Since by Theorem \im N _^ oo ip N (0) = ij)(Q), solving this differential inequality and then 
letting N — > oo and E\ — > implies that 

lim ip N (t) = ip(t) for t < t - (6.21) 

The derivatives ^'(t) and <p' N {t) are both bounded, which is apparent from (j6.12|) and (|4.1()j) . 

and we get 

IimBup|^(l)-V(l)| <K{l-t ). 

N-*oo 

Using lj?H7l) . 

limsup|y^(l) -P(£,w)| < — t ) + e 
and letting e — > and to — ► 1 finishes the proof of Theorem ^ 

□ 

Proof of Theorem 03 The proof of the the first part follows from Theorem[7|in exactly 
the same way as Theorem ^ follows from Theorem El The uniqueness of (q, A) follows from 
the following simple argument. (j6.21j) implies that for t < t 

lim / x 1 (( J R 1 , 2 -g) 2 ) = 

and since the definition of /ii does not depend on A this q must be unique. Since Vi(q, A) is 
convex in A, this implies the uniqueness of A. 

□ 

7 Computing global free energy. 

In this section we will prove Theorem |21 which will follow from Theorem ^ and Gaussian 
concentration of measure. Let us start by proving the following concentration inequality. 

Lemma 8 For any measurable subset Q C T, N let us consider a r.v. 

X = log / expH N {tr)du{a). (7.1) 
Jn 

Then, for any t > 0, 

P(\X -EX\ > 2v / ZiVt) < 2exp(-t), (7.2) 
where L = max{£(x) : x G [d, D]} + c(N). 
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Proof. The proof is a simple modification of Theorem 2.2.4 in [H]. Unfortunately, Lemma 
IHJdoes not fall into the framework of Theorem 2.2.4 in jH] directly, but the same argument 
still works if we utilize the particular definition of X and the covariance structure of 
the Hamiltonian Hn((t). 

Let H\j and Hfj be two independent copies of the Hamiltonian H^. For t G [0, 1], we 
define, 

Fj = log / exp(VtH J N (a) + VT=tH N (a))dv((r) (7.3) 
Jn 

for j = 1, 2 and let F = iq + F 2 . For s > 0, let 

p(t) = Eexps(F 2 -iq). 

If we define 

^(<r\ <x 2 ) = ^{H^a 1 ) + tfjUO) + VT^tiH^a 1 ) + ^( CT 2 )) 
then straightforward computation as in Theorem H] gives 

<f/(t) = -iVs 2 Eexps(iq-F 2 )exp(-F) / C("-\ cr 2 ) expff^cr 1 , <r 2 )dz/(o- 1 )dz/(o- 2 ), (7.4) 

where £ was defined in (14. 14(1 . Since |C( crl , cr 2 )| < L, we get 

(p'(t) < LNs 2 Eexps(F 1 - F 2 ) exp(-F) / exp H t dudu = LNs 2 (f{t). (7.5) 

By construction </?(0) = 1, so that (J7.5|) implies that ip(l) < exp NLs 2 . On the other hand, by 
construction, <p(l) = Eexp s(X — X'), where X is defined in (17. 1|) and X' is an independent 
copy of X and, thus, 

Eexps(X -X') < exp NLs 2 . 

By Jensen's inequality, this implies that, Eexps(X — EX) < exp NLs 2 and using Markov's 
inequality we get that for t > 0, 

P (^X — EX > tj < inf exp(NLs 2 - st) = exp(-^— 

Obviously, a similar inequality can be written for EX — X and, therefore, 

P(|JT-K|>t)<2«p(-Jl). 

This is equivalent to (|7.2j) . 

□ 

The proof of Theorem |21 will be based on the concentration inequality of Lemma |H1 and 
the following result. 
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Theorem 8 For en = N 1 there exists a set {u±, . . . ,Um} Q [d, D] of cardinality M < LN 
such that 

[d, D] C (J [m - e N , Ui + e N ] (7.6) 

i<M 

and such that for all i < M, 



F N ( Ui ,e N )< sup V{£,u)+K, (7.7) 

d<u<D 



where \TZ\ < + Lejq. 



We will first show that Theorem El follows from Theorem ^ and Theorem |H1 

Proof of Theorem 2. It follows from the definitions that for any d < u < D and any 

sequence (ejv), F n {u,Sn) < F N and, therefore, Theorem [T] implies 

liminfF/v> sup V(£,u). 

N^oo d<u<D 

To prove Theorem El we need to show that 

limsupF/v< sup V(£, u). 

N^oo d<u<D 

Suppose not. Then for some e > 0, 

lim sup F N > sup u) + e. 

Af^oo d<u<D 

To simplify the notations, instead of considering a subsequence of N we simply assume that 
for iV large enough we have 

F N > sup V(£,u) + e. (7.8) 

d<u<D 

On the other hand, let = N' 1 and consider a set of points {ui, . . . ,um} as in Theorem 
IH1 so that for iV large enough for all i < M, 

F N ( Ui ,e N )< sup ?(^«) + ^. (7.9) 

d<u<D £ 

Lemma |H1 implies that for any t > with probability at least 1 — LN exp(—Nt 2 /4L) we have 

Fn < log Z N + t and — logZ N (ui, e N ) < F N (m, e N ) + 1 (7.10) 

for all % < M. Therefore, (|7.8j) . (|7.9|) and (|7.10p imply that for iV large enough with probability 
at least 1 — LN exp(—Nt 2 / AL) we have 

— log Z N ( Ui ,e N ) <F N (u,e N )+t<F N + t-^< — logZ^ + 2t - |. 

Taking t = e/K we get that for N large enough with probability at least 1—LN exp(—Ne 2 / K) 
we have 

Ne 

\fi < M, log Z N (ui, e N ) < log Z N - —. 
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This yields that for i < M the Gibbs measure 



G N ({R 1A E [ui - e N , Ui + e N }}) < exp 




and, therefore, by (|7.6)L 

1 = G N (Z N ) = G N ({R 1A e [d,D]}) < LiVexp(-^), 
which is impossible for large N. 

□ 

The statement of Theorem |S] is intuitively obvious considering ([5.6)1 . We, basically, need 
to show that the term mat (A) in ()5.6|) can be substituted by u in a controlled manner. This 
will be based on three technical lemmas. To formulate the first lemma, it will be convenient 
to think of the pair (m, q) in terms of the function m = m(q) defined in ()A.32|) . 

The following Lemma is the analogue of a well known continuity property of X (m, q, A) 
with respect to the functional order parameter m(q) in the SK model (the statement can be 
found in 2 and the proof is given in ^3]) and since its proof is exactly the same we will not 
reproduce it here. 



Lemma 9 For any A and for any functions m(q) and m'(q) defined by HA . 32^ and corre- 
sponding to pairs (m, q) and (m', q') we have 

\X {m,q,X)-X {m',q',\)\<L [ \m{q) - m'{q)\dq, (7.11) 

Jo 

where the constant L depends on £ and D only. 

Next, we will describe several properties of the function 7\.(m, q, A, u) defined in ()1.15|) . 
For any k, any vectors m, q and any d < u < D let \{u) be defined by (|5.19|) . 

Lemma 10 For any 5 > there exists a constant A(5) such that for any vectors m, q and 
any u G [d + 5, D — 5] we have 

\X(u)\<A(5). (7.12) 

Proof. For a fixed m and q minimizing Vk(fn, q, X,u) over A is equivalent to minimizing 
—Am + X (m, q, A) over A. By Lemma 

—Am + X (m, q, A) > A(a(A) — u), 

where a(A) satisfies (j5.9|) . Therefore, for any 5 > the right hand side goes to infinity 
uniformly over d + 5<u<D — 5 and, thus, the left hand side goes to infinity uniformly 
over m, q and d + 5<u<D — 5. This, obviously, implies that there exists A (S) such that 
(17121 holds. 

□ 
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Lemma 11 There exists 5 > such that for all m and q we have 

X{u) < for uE[d,d + 5), X{u) > for u G (D - 5, D] . (7.13) 
Proof. Using (j3~T5jl and (jCTjl . 

dX 

-^-=EW 1 ...W k (a 2 )=u, (7.14) 
where for a function /i(er), (h) is defined by, 



expX fc+1 = J h{a) exp^a z p + \a 2 ^dv{a). (7-15) 



Since X is convex in A, dX /d\ is increasing in A and, therefore, ()7.14|) implies that A = X(u) 
is nondecreasing in u. Therefore, in order to prove (|7.13|) . it is enough to show that for some 
5 > if X(u) = then u G [d + 5, D — 8]. We will only prove that X(u) — implies that 
u > d + 5 as the case u < D — 5 is quite similar. 

We set A = and denote Z = J2 P <k z v Given 7 > 0, we can write 

¥W l . . . W k (a 2 } > EWi . . . W k (a 2 }I(\Z\ < 7). (7.16) 

First of all, let us show that if \Z\ < 7 then 

(^ 2 )>rf+^exp(-L 7 ), (7.17) 

where a constant L does not depend on 7 (but it depends on other parameters of the model 
such as measure v). To show this, let us first note that \i\Z\ < 7 we have 



expX fc+1 = / exp(aZ)du(a) < Lexp(L'y), (7-18) 
is 

using (jl.7|) . Next, 

(cr 2 )expX fc+1 = J a 2 exp(crZ)dv(o-) 

= dexpX k +i + J {c 2 — d) exp(aZ)dis(a) 

> dexpX k+1 + — exp(-iyy), (7.19) 

where the last inequality is obtained by restricting the integral to the set {a 2 G [(D+d)/2, D]} 

of positive measure v by (|1.7jl . Combining (|7.18jl and ()7.19j) yields (|7.17jl . Plugging (|7.17jl 
into (|7.1fj|l implies 

W x ...W k (a 2 ) > (d+-exp(-L^)^EW 1 ...W k I(\Z\ < 7) 

= (d+^exp(-L7))(l-EW 1 ...W fc /(|Z|>7)). (7.20) 
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Using Holder's inequality, we can bound 

EW l ...W k I{\Z\> 1 ) < (F(\Z\> 1 )) 1/2 (E(W 1 ...W k ) 2 ) 1/2 

< L^^Y^iw^^w^yi^ (7.21) 

since Z is a Gaussian r.v. with variance uniformly bounded for all d < u < D. As in 
()4.11|) we can write 

W 1 ...W k = expX fc+1 J](exp(-X,)) mi - m! - 1 

Kfc 

and by Holder's inequality, 



nWi...W k ) 2 < (Eexp(4X fc+1 )) 1/2 n( Eex P(- 4 ^)) (m " mi ~ l)/2 - (7-22) 



The first factor on the right hand side can be estimated as follows, 

Eexp(4X fc+1 ) = exp(aZ)du(a)Y <E J exp(4 : aZ)du(a) 

exp(8a 2 £'{u))dv{a) < L. 



Next, since 

Xi = — log E/ exp mp^+i > E t Xi +1 , 
mi 

repeating this over I yields that Xi > EiX k+ i. Therefore, 

Eexp(-4X ; ) < Eexp(-4EiX fc+1 ) < E exp(-4X fc+1 ) = E^ exp(aZ)du(a) ) 

< E / exp(-AaZ)du(a) = / exp(8a 2 £'(u))dv(a) < L. 

Plugging all these estimates into ()7.22|) gives 

E(Wi . . . W k f < L 

and ()7.21|) implies 

2 

EW 1 ...W k I(\Z\> 1 )<Lexp(-? ! -y 

Finally, (TH^) implies that 

u = EW 1 ...W k (a 2 ) > (d+ -jr exp(-L 7 )) (l -Lexp(- 

> d + — exp(— I/y) — Lexp^— 



L 

2 



7_ 
L 

Taking 7 large enough gives w > d + 5 for some 8 > 0. As we have already mentioned above 
one can similarly show that A = implies that u < D — 5 for some 5 > and this finishes 
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the proof of Lemma. 

□ 

We are now ready to prove Theorem |HJ 

Proof of Theorem El Let us start with equation (|5.fi)l that states that for any A, m 
and q such that qk+i = u, 

F N {u,e N )<-Xu N {X)+X {m,q,X)-- E m,(0(g,+i) - %,)) + K, (7.23) 

i</<fc 

where mat(A) is defined in (|5.3|) and \TZ\ < cn + Lew- Let us note that the definition ()5.3|) 
implies that 

|wjv(A) -w| < e N . (7.24) 

Let us take 5 > as in Lemma ^2 Lemma E3 implies that for all d + 5 < u < D — 6 we 
have |A(w)| < A(5). Moreover, (T7T23)) and (|TT23I) imply that for any <i + 5<w<D-5 and 
|A| < A(5) we have 

F N (u, e N ) < -Xu + X (m, q, A) - - E m t (0(ft + i) - %*)) + A( 5 ) £ ;v + ft 

l<l<k 

= V k {m,q,X,u)+K, (7.25) 

where again \1Z\ < cn + Len- The fact that for d + 5 < u < D — 5 we have |A(w)| < A(8) 
means in this case that for a fixed (m,q) minimizing Vk{Tn,q, X,u) over A is equivalent 
to minimizing it over |A| < A(S) and, therefore, minimizing Vk(fTi, q, A, u) over (m,q,X) is 
also equivalent to minimizing it over m,q and |A| < A(S). Therefore, ()7.25|) implies that for 
d + 5<u<D-5, 

F N {u,e N ) <V{£,u)+K< sup V{£,u)+K, (7.26) 

d<u<D 

where \TZ\ < + Le^. 

Next, let u be such that u + < d + 5. Let us consider arbitrary k' > 1 and arbitrary 
m! and such that q' k , +l = u N (X). By (|7.24|) . there exist m and q, maybe, with different 
parameter k, such that q k+ i = u and 

|m(g) - m\q)\dq < \u — u N (X)\ < e N , (7.27) 

where functions m(q) and m'(q) are defined in ()A.32j) in terms of (m, q) and (m', qr') corre- 
spondingly. This can be achieved by simply assigning m(q) = 1 for q between u and un(X) 
and, otherwise, letting m(q) = m'(q). Then Lemma El implies that 

\X (m,q,X) - X (m',q',X)\ < Le N . 

Also, obviously, condition ()7.27|) implies that 

l \ E E ^(%; + i)-%D) <^ 



KZ<fc KKfc' 



AT- 
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Therefore, (|7.23|) implies that for any k',m' and q' such that q' k , +1 = un{X), 

F N (u,e N ) < -Xu N (X)+X (m',q',X)-^ ]T m'Mq'^) - 6(q[)) +TZ 

l<Kfc' 

= V v (rri,4,\u N (\))+K, (7.28) 

where again \1Z\ < cn + Len- Since now u + £tv < d + 5 < D, for A < the definition (|5.3jl 
implies that mat (A) = it + and, therefore, for A < we have 

F N (u,e N ) <V k >{m',q',X,u + e N )+n. (7.29) 

Since u + En < d + 5, Lemma ^2 implies that for any m' and q', X(u) < which means that 
for fixed (m f , q') minimizing Vk'(fn', q', X,u + En) over A is equivalent to minimizing it over 
A < and, therefore, minimizing 7V(m/, q 1 , A, u + en) over (m', q 1 , A) is also equivalent to 
minimizing it over m', q' and A < 0. Hence, (|7.29j) yields that it u + En < d + 5 then, 

F N (u,e N )<V{£,u + e N )+K< sup V(£, u) + TZ. (7.30) 

d<u<D 

Similarly, one can show that this holds for u such that u — en > D — 5 and, combining this 
with (|7.26j) . we showed that for all u in the set 

{u + e n < d + 5} U {d + 5 < u < D - 5} U {u - e n > D - e n } (7.31) 

we have 

F N {u,e N ) < sup V(£,u)+K. 

d<u<D 

It is obvious that for en = N' 1 the set ()7.31|) contains e^-net of the interval [d,D] of 
cardinality LN. This finishes the proof of Theorem |HJ 

□ 
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this research. 
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A Proof of the apriori estimates. 

We will prove the apriori estimates of Section |H1 in several steps. In Section lA.ll we obtain the 
analogue of Talagrand's interpolation for two copies of the system that is the main technical 
tool of the proof. In Section IA.21 we summarize several properties of the parameters in the 
definition of the Parisi formula in ()1.15|) and ()1.16|) and in Sections IA. 31 and IA.4I we prove the 
apriori estimates of Section |B1 by considering two separate cases of "far" points and "close" 
points. 

A.l Talagrand's interpolation for two copies. 

The key to proving the apriori estimate of Section H is Talagrand's interpolation for two 
copies of the system. The proof of this result is similar to the proof of Guerra's interpolation 
in Theorem 0] Once the Talagrand's interpolation is obtained, the arguments in the rest of 
the paper will be adapted from ^T] with some necessary modifications. 

For v G [-D, D], let v = rj\v\ for rj = ±1. Consider k > 1 and consider a sequence n : 

= n <n 1 <...<n K = l (A.l) 

such that n T = \v\ for some r < k. Consider a sequence p : 

= po < • • • < Pk+i = u. (A.2) 
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Consider a sequence of pairs of random variables (Y p ,Y p ) independent for < p < k such 
that 

E(F p 1 ) 2 = E(F p 2 ) 2 = t(£'(p p+1 ) - ?(p p )) (A.3) 

and such that 

Y p = T]Yp for p < t and Y , YJ 2 are independent for p > t. (A. 4) 

Let (Z p , Z p ) be an arbitrary sequence of independent vectors for < p < k. Let (Zf , Z 2 p ) 
and (Yl p ,Y? p ) be independent copies for i < N of the sequences (Z p ,Z p ) and (Y p ,Y p ) and 
we assume that they are independent of each other and the randomness in the Hamiltonian 
Hn(<t). For s G [0, 1], let us define an interpolating Hamiltonian 

H s {a\ <r 2 ) = J7tH N {* 1 ) + VTtH N (<r 2 ) + E E a * E (%p + ( A - 5 ) 

j<2 i<N p<K 

and consider 

F = log / expi/ s (o- 1 ,cr 2 )rfz/( C r 1 )rfz/( ( T 2 ), (A.6) 

JA(v) 

where A(v) was defined in ()04j) . Let 

X (s) = ±KP(n)F. (A.7) 



Theorem 9 For s G [0, 1] we have 

x'(s) < -2tJ2Md(pi + i)-d(pi))-tzZ n ^P^- 9 ^ + n ( A - 8 ) 

1<T 1>T 

where \K\ < K(e N + c(N)). 

Proof. The argument is similar to Guerra's interpolation in Theorem |3J 1)3.15)1 implies that 

1 dF 
x'(s) = -E Wl ...W K — , 

where 

Wi = expxn(F l+1 - F t ) 
and where Fi are defined as in (J3.2)) . Using the fact that n K — 1, one can write 

W 1 ...W K = exp E n,(F, +1 -F,) 

1<1<K 

= exp (F + E ( n '-i - n i) F i) =TexpF (A.9) 

1<1<K 
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where T = Ti . . . T K and TJ = exp(n/_i — ni)Fi. Using the definition of F in (|A.6 



1 dF 
X '(s) = -ETexpF— = l-ll 



where 

v7 



and 



1=-^ f ETiH^a 1 ) + H^cT^expHsicT 1 ,* 2 ^* 1 ^* 2 ). (A.10) 
2AV S Jam 



II- I / ET^^a^^exp^^ 1 ,^)^ 1 )^ 2 )- (A-H) 

/ - 1 - — s J am ; x»r 



j<2 i<N p<K 



To simplify the notations, let us denote cr = (cr 1 ,^ -2 ) and p = (p^p 2 ), let dv(cr) = 
dv(er 1 )db'((T 2 ) and define 

3(<r,p) = ^E(^( < T 1 )+^( < r 2 ))( J ff iV (p 1 ) + i7 Ar (p 2 )) 

= C^^ + C^.^+C^^ + C^A 2 ) (A.12) 

where £ was defined in (14.14)1 . To compute I, we will use (|4.9)) for the family 

f/(p) = H^p 1 ) + ^(p 2 ) for p = (p\ p 2 ) G 

and we will think of each factor Ti in T as the functional of g(p). Then flOJ) and (lA~T2l 
imply, 



I = / ET^^.^M (A.13) 

<9exp H s (cr) 



'AM 

ETexp^^I^b^p)]^)- 
Jam ?! £g 



First of all, 

/stexpHJcr 
dg(cr) 

and, therefore, the first line in (|A.13|) can be written as 



l - [ ETexpH a (<T)t(tr,a)dv(o-) = lEW 1 ...W K {3(<T,, 
1 Jam 1 

where 

' "' ' ' <• > ///(cr). 



(A.14) 



($((T,cr)) = exp(-F) / 3(0-, cr) expH s (cr)d 

JA(v) 

Using the definition of T\ we can write 

i^[3(<T,p)] = ( nw -n0^[3(cr,p)] = (n^ - n,)W, . . . wj^h(tr, p)], 
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where we used (13.16)) . We have 

SF f 

■r-\}(<r,p)] = Vsiexp(-F) / 3(0-, p) exp H s (p)du(p) =: Vsi($((T, p))', 
og Ja(v) 

where (•}' denotes the Gibbs average with respect to p for a fixed cr. Therefore, for a fixed 
<x we get 

1 ST 

where 7; here is defined by analogy with (|4.5|) . Combining this with the fact that 

TexpH s {(T)du{a-) = Wi . . . W« exp(-F) exp H s {cr)dv{er) 
we can write the second line in ()A.13j) as 

lY J (ni-i-n l )EW 1 ...W K exp(-F) [ 7,(3(0-, p))dv{cr) (A.15) 
= \ - • • • ^-i7f 2 (3(^, P)) = I X>_i " "Owfcfo P)), 

1<K 1<K 

where /i; here is defined analogously to ()4.6|) . Combining ()A.14|) and ()A.15|) we get 

I = t -EW 1 . . . W K (i{<r, cr)) + t - - mMi(<r, P))- (A.16) 

1<K 

Let us denote by Rjji the overlap of tr 7 and tr 7 ' and by R?^ the overlap of cr- 2 and p- ' . From 
(fOl and (lA~T2l) . 

i = ^;ir, . . . w K ( £ e(^0> + 5 £>-i " n <)M E tf^)) + ^> ( A - 17 ) 

where |7?.| < 4c(iV). Since the average (•) is over the set A(v), we have 

\Rj,j — u\ < e N and |i2 lj2 —v\<-^. (A. 18) 

Therefore, 

I = *(£(«) + + I " "J)W(Z) + ^, (A.19) 

l<k j,j'< 

where \TZ\ < K(en + c(N)). The computation of II is very similar. For p < k, let us define 

j<2 i<N 
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so that II = ^2 p<K H{p) where 

II(p) = — \ / ~ETg p (<r) exp H s (<r)du(<T) 

2Ny/l - s Jam 



Let us define 

h (a, p) = ±Eg p (<T)g p (p) = £ RU'EY'Y'' . (A.20) 

i,i'<2 

Then one can repeat the computations leading to (jA.lfijl with one important difference. 
One needs to note that F\ does not depend on the r.v. Y i;P for I < p and, as a result, the 
summation in the second term will be over p < I < k, i.e. 

II(p) = \eW 1 . . . W K ( h (<r, er)) + \ Yl ( n '-i " *Mh(°> P)) (A.21) 

and 



2 

p<l<K 



ii = i:-:ir, . . . *)) + ~ E^ - n^(E^°"> p)) (A.22) 

p<K (<re p<Z 

Using ()A.18|) and ()A.3|) . it is easy to see that the first term on the right hand side is 

tu£(u) + t\v\£(\v\) + n. 

Using (lA~3j) and (|A~4|l . 

^E(r/) 2 = te'(ft) and J] Ey p ly P = ^'(ftAr) = t£' {vQiat) 1 
p<i p<i 

where in the last equality we used the fact that £ is even function. If we define 

gf J = qi and g, 1,2 = ^ Ar (A.23) 
then ^ p<l E7/y/ = t£'(gf i ') and |A~22jl can be written as 

n = tt^(«)+>|?(M) + i3^ (A.24) 

!<« j,i'<2 

Since 6*(x) = x£'(a;) — £(x) is also even, (|A.19J) and (|A.24|) imply that 

x '( s ) = i-n = -te{u) - te(v) + \ J> - n^-x) Yl e ^' j ') + n 



0,3 



1<k j,j'<2 

- \ E( n ' - n >-^ ( E (^') - Rj ' j 'z'(vi' j ') + 

< t9(u) - te(v) + ~ J> - m_0 £ W J ") + ^ 

i,i'<2 

= tQ{u) - tB(y) + £ 0^) " ^ E E " + ^ 

j,j'<2 i,i'<2 i<k 

= iEE^€)-%?' J '))+^ 

j,i'<2 / r 
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since = u and = \v\. To finish the proof it remains to use (|A.23j) . We have 
E ( 9 ^ix) - = 2(%m) " %/)) for I < r 

j,j'<2 

and 

E - ^^')) = - 6( qi ) for / > T , 

since for / > r, - 6»(^' 2 ) = % T ) - % T ) = 0. 

□ 

We can bound x(0) as follows. Using ()A.18|) . for any AjGl, 
^| s=0 < -2NXu- N~/v + 2N\X\e N + \j\ 

+ log f ex P E(E^E(^+ r i) + E A ^) 2 +^ v O dz/(CTl)dzy(<T2) 

= -2NXu - Njv + 2JV| A|e jv + |t| + E 

i<N 

where Fj(A, 7) are independent copies of 

F(A, 7 ) = log / exp(E^E(^ + F /)+E A ^) 2 + ^ lCT2 )^ (cTl) ^ (0r2) - (A ' 25) 

(jS3J), (Q and (jSHJ) now imply that 

X(0) < -2Aw - -fv + 2|A|e;v + iTliV -1 + P(n)F(A, 7) (A.26) 
and we obtained the following. 

Corollary 2 If x( s ) an d F'(X, n f) are defined by \A.7p and lA.2fy) then for all A, 7 6 R, 
X (l) < -2\u ~ 1V + V(n)F(X, 7) - 2t E M°(pi+i) ~ ~ * E n <(%m) - 9(pi)) + n 

1<T 1>T 

(A.27) 

where \K\ < K{e N + |A|erjv + + c(N)). 

Proof. The proof follows immediately by combining (jA.8|) and (|A.26[) . 

□ 

Remark. The remainder TZ in Theorem |H1 will be a result of application of (jA.27|) . We 
will use it for A = X(u) defined in ()5.20|) and, as in the proof of Lemma 13 |A(w)| < A for 
some A that depends only on £, v and u. Below we will use flA.27|) for I7I < L for some 
constant L independent of N,t and v. As a result, the remainder in Theorem |Hl \TZ\ < 
for a N = K(e N + c(N) + N' 1 ). 
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A. 2 Properties of e-minimizer. 

In this section we will describe several properties of the sequence (k, m, q, A) in (j6.13|) that 
follow from its definition. Section 4 in ^T] describes these properties in a very general setting, 
with no reference to the classical SK model and all the computations there apply to our case. 
The only difference is that in ^1] certain generic computations were applied to the function 
log chx and here we will apply them to the function 

x -> log J exp(ax + Xa 2 )du(a). (A.28) 

We will not reproduce some of the generic computations in [TT] that directly apply to our 
case. 

Perturbing the sequences m and q. Let 

Qr-i < a < (Zri b = £'(a) and m r -\ < m < m r 

and define new sequences m', q' by inserting m and a into m and q. Let us consider a 
sequence (z') p <k+i of independent random variables such that E(z p ) 2 = C(lp+i) ~~ CWp) or ; 
expressing this explicitly in terms of b, 

E(4) 2 =e'(g P+ i)-e%)forp<r-l, 
Eiz'^^b-eiqr-i), E(z' r ) 2 = ?(q r )-b, 
W P+ if = C'fe+i) - e'(?p) for r < p < fc. 



Let 



F = log y exp^er ^ + \a 2 ^dv{(j) 



p<k+l 

and consider functions 

T(m,b) =V(m')F (A.29) 

and 

$(m,a) = -A M + r(m,r(a))-i^m / (0(g i+1 )-0(g / ))-^(m-m r _ 1 )(0(g r )-0(a)). (A.30) 

z<fc 

Comparing with the definition ()1.15|) it is clear that 

T(m, £'(a)) = X (m', A) and $(m, a) = V k+1 {m', q' , A, w) (A.31) 

and, thus, T and $ describe the behavior of Xo,Vk when we perturb the set of parameters 
by adding an extra point. It will be very convenient to note that the functionals Xq and Vk 
depend on the sequences m, q only through the function m(q) defined by 

m(q) = mi for q x < q < q i+1 , (A.32) 

which is called the functional order parameter. Therefore, inserting parameters m and a can 
be visualized as the perturbation of m(q), as shown in Figure El From this point of view, it 
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m r _i 



m 



q r -i ~a q r +i 
Figure 3: Perturbing the e-minimizer. 



becomes obvious that for all m and a as above, 



TK_ 1; b) = T(m, e(q r )) = X (m, q, A), 
$(m r _i, a) = $(m, q r ) = V k {m, q, A, u) 



(A.33) 
(A.34) 



and, thus, it is very important to study the behavior of the derivatives of T, $ in m and a 
at m — m r -i and a = q r . 

Properties of the derivatives. Let us define 



U(b) 



dT 
am 



m=m, — i 



and 



<9$ . 

— (m, a ) , 

GUI 1 m=m r -\ 



\u{aa))-\{e{q r )-B{a)). 



The first fundamental formula is 

dT 



( )b ^ m ' 6 ^=C'(3r) 



1 



m — m r _i)A 



(A.35) 
(A.36) 

(A.37) 



where A is independent of m. This can be obtained by a straightforward computation and 
one can write down an explicit formula for A (see jTTj) but we will omit it here. By definition 
of the e-minimizer ()6.13jh (m, q, A) is the minimizer of Vkijn, q, A, u) and, therefore, 



— — = — h -(m r - m r _i)g r ^ (g r ) = 0, 



(A.38) 



where we used (jl.l5j) and the fact that 9'(q r ) = q r £"(qr)- When m = m r , it is apparent 
from Figure El that the intervals [a,q r ] and [q r , q r +i] are glued together in a sense that all 
functionals defined above become independent of q r and, in particular, using (jA.31|) 



T(m r ,£'(a)) = X (m' , q' , \)\ m=mr = X (m, q, A)| 9r=0 . 
Taking the derivative of both sides with respect to a at a = q r gives 



(A.39) 



— (m, q ,X) = -(m r ,Ha))\ a=gr 



dT 



b= S>(q r ) = ~ mr-l)Z"{qr)A 
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where the last equality follows from (|A.37|) for m = m r . Comparing this with ()A.38|) implies 
the first consequence of (|6.13p . 

A = q r . (A.40) 

Next, (lA~35j) and (|A~37|) imply that 

) = JL(!*L( m ,b) ) =--a, 

where, given any doubt, equality in the middle can be checked by computing both sides. 
Therefore, 

-tf'(£'(9r)) — A — q r . (A.41) 
Another crucial property of U(b) that can be verified by straightforward computation is 

U"(b) < 0, (A.42) 

i.e. U(b) is concave in b. Next, let us describe several properties of f(a) in (|A.36j) . We have 

f(q r ) = fW = 0, f(q r -i) > 0. (A.43) 

The first one follows from 



f{qr) = \u{aqr)) = ^{m^\q r )) 



0. 



since (|A.33|) yields that T(m,£'(q r )) does not depend on m. The second one follows from 
(lA~36|) and (lA~4T| since 

f'(qr) = \e'(qr)(U'(e(q r )) + q r )=0. 

To show that f(q r -i) > let us note that 

$(m,g r _i) =V k {m, q, A, u)\ mr _ 1=m 

since setting a = q r _i simply replaces m r _i with m in the definition of Vk, which is also 
apparent from Figure El If m T -\ > 0, then as in (|A.38[) . 

— = o =► /( fr _o = ^K^OU^., = o. 

If m r _i = then since (m, A) is the minimizer of Vk, slightly increasing m r _i should not 
decrease Vk and, therefore, the right derivative dVk/dm r ^i > and f{q r -i) > 0. 

e-dependent properties of the derivatives. So far we have only utilized the fact 
that (m, q, A) is the minimizer of Vk and we have not used the condition in (j6.13|) that 

V k (m, q, A, u) < V{£, u) + e. 

In particular, this implies that for any m and a, 

$(m r _i, a) = "P fe (m, q, A, u) < u) + e < $(m, a) + e, (A.44) 
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which means that we can not decrease Vk much by varying parameters m, a. This can be 
combined with the following fact that plays a central role: 

All derivatives of T, $, U, f with respect to a, b, m are bounded by constants 
that depend only on £, u, u but not on (k, m, q). (A. 45) 

Let L denote such constants that depend only on £, u, u. The proof of (|A.45|) in [TT] relied 
on the fact that Ech(c + c'z) L < L for a standard Gaussian z and |c|, \c'\ < L. In our case, 
this conditions will be replaced by an obvious condition, 

j exp(acz + \a 2 )dv(a)^ < L for |c|, |A| < L. 

The following Lemma holds. 

Lemma 12 The junction /(a) in \A . 36\) satisfies, 

/(a) > -Ly/e (A.46) 

and 

f"(q r ) = (e(qr)U"(?(q r )) + l) > -Le 1 " \ (A.47) 

Proof. ()A.46jl holds if /(a) > so we can that /(a) < 0. Using ()A.3fijl and (|A.45)I . we can 
write 

$(m, a) < $(m r _i, a) + (m - m r _i)/(a) + L(m - m r _i) 2 . (A.48) 
By ()A.34|) and the fact that (m, q, A) is a minimizer of Vk, 

$(m r _i,a) =Vk(m,q,\,u) < V k (m, q, X, u)\ qr=a = $(m r ,a), 

and the last equality can been seen as in (lA~39l) . Therefore, (lA~48l) with m = m r implies that 

(m r — m r _i)/(a) + L(m r — m. r _i) 2 > 

and, therefore, 

m r > m r _! — ^ > m r _i, (A. 49) 

where the second inequality follows from our assumption that f\a) < 0. ()A.44|) and ()A.48jl 
imply that for any m and a, 

—e<(m — m r _i)/(a) + L(m — m r _i) 2 . 

Taking m := m r _x — f(a)/2L, which belongs to [m r _i,m r ] by (|A.49|) . implies 

s < -f(a) 2 /AL 

and this proves (jA.46|) . It remains to prove ()A.47|1 . From ()A.36|) and ()A.41|) we see that 
f'(qr) = \e\qr) (U'(e(q r )) + q r ) + ^'W {11"^))^) + l) 
= le(qr)(U"(?(q r m"(qr) + l). 
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(|A.45|) and (|A.46|) imply that for any a, 



< /(a) < -f"(q r )(a - q r f + L\a - q r 



(A.50) 



Using (jA.43|) we have 



0<f(q r -i)<-f"(qr)(qr 



i - qr) 2 + L\q r -i - q r 



and, hence, 



q r > g r _i 



/%■) 



2L 



If f"{q r ) > then tOTj) holds, otherwise 



a = q r -i ^y— e [g r _i,g r ] 




and using ()A.50|) for this choice of a again implies ()A.47|) . 



□ 



Dual construction and the replica symmetric case. 

The construction above will be used in the proof of Theorem El to provide control of the 
points on the left hand side of q r , i.e. g r _i < v < q r . In order to provide control of the points 
on the right hand side q r < v < q r+ i one can consider a dual construction by perturbing 
parameter m r on the interval [Or,a]. This construction is very similar so we will not detail 
it and we will only consider the points on the left hand side in Theorem |H1 In the replica 
symmetric case, the function $(m, a) defined in (J2.7)) is the analogue of the function in ()A.30j) 
and the properties (j2.10|) and (|2.11|) replace the properties (jA.46|) and (jA.47|) and, in fact, 
are stronger because e is replaced by 0. (The change of sign in the inequalities is simply 
because we consider a dual construction.) Therefore, the proof of the replica symmetric 
apriori estimate in Theorem is exactly the same as the proof of Theorem El if we use (j2.1()j) 
and (|2~TT|) instead of (jA.46j) and fjA.47jl . and we will not detail it. 

A. 3 Control of the far points. 

In this section we will prove the easiest case of Theorem |H1 when the point v is far from q r in 
the following sense. Given e > 0, let (k, m, q, A) be an e-minimizer defined by (16. 13)1 . Without 
loss of generality, we will assume that all coordinates of the vector m are different and that 
all coordinates of the vector q are also different. Otherwise, we can decrease the value of k 
by gluing equal coordinates without changing the value of the functional ^(m, q, A). In this 
section we will consider the case when v E [-D, D] is such that 



v < q r -\ or v > q r+ \ if r > 2 
v < —qi or v > c/2 if r = 1 



(A.51) 



and we will prove the following. 
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Proposition 2 In the notations of Theorem® if hA.51\) holds then 

—WP{n)F(A{v)) < 2i/)(t) - K + 11, (A.52) 

where K > is a constant independent of N, t and v. 

This implies Theorem E] in the range of parameters (|A.51J) because 

K>(v- q r ) 2 /(K~ l (q r - q r ^f A (q r+1 - q r ) 2 ). 

The proof of Proposition 121 is based on the following three step construction. 

1. Let us recall the definition of n in (|6.4j) and (z p , z 2 ) p <k in 

2. [Inserting \v\.). Let a be such that 

q a < \v\ < q a +i- (A.53) 

Consider a vector 

q' = (q' , ■ ■ q'k+2) = (io, • • • , q a -u \v\,q a , • • • , qt+i) 

which is defined by inserting \v\ in the vector q and define a vector 

, m m a _i 
n = (— , . . • , — ^— , m a , . . . , m fc ). 

Consider a sequence (2/ p ) p <fc+i of independent Gaussian random variables such that 

%J = £'(g; + i) - W P ) 

and let {y^y 2 ) p <k+i consist of two copies of {y p ) p <k+i such that 

2/p = rjUp for p < a and y^, y| are independent for p > a, (A. 54) 

where v = r)\v\. 

3. (Gluing two sequences together). Let k — 2k + 1. Let us consider a vector 

n = (no, • • • , n«+i) such that n < . . . < n K+ i 

and such that n consists of the elements of vectors n and n'. More precisely, there exists a 
partition /, J of the set {0, . . . , k + 1} with card/ = A; + 1 and cardJ = k + 2 such that the 
elements n p are the elements of n for p & I and the elements of n' for p E J. For < p < «, 
define 

(Zp 1 , Z p 2 ) = for p e J and (Z p x , Z 2 p ) = v / T 3 t(4, 2?) for p E I (A.55) 
and where Z is such that n p = rt\. Similarly, define 

(K 1 , Y 2 ) = for p e I and (Y 1 , Y p 2 ) = Vt(yj, yf) for p e J (A.56) 
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and where I is such that n p = n[. For < p < k, let us define 

(g 1 p ,g 2 p ) = (Z l p ,Z 2 ) + (Y p \Y 2 ) (A.57) 

or, in other words, 

(<?J, g 2 p ) = (Zi, Z 2 ) for p G / and = (l^ 1 , F p 2 ) for p E J. 

In order to match this definition of (YJ) with (jA.2|) and ()A.3|) . let us define a sequence 

Po < • • • < Pk+i 

such that p p = q[ for I such that n p = n[. Define r by n T = n' a = m a _i. 

We will now apply Corollary to these choices of n, (Z-j) and (1^). First of all, 

X (l) = ^EP(n)F(A(t;)) (A.58) 

since for s = 1, the random variables (Yl) will disappear in the definition of H s (a l ,(T 2 ) in 
(|A.5J) . the Hamiltonian H s (a l ,cr 2 ) will coincide with Hamiltonian H^tr 1 , <r 2 ) in (|fj.2jl and, 
as a result, the definition of F in (jA.6|) will coincide with F(A(v)) in Theorem |H1 Next, it is 
clear from the construction that 

-2t M^Pi+i) ~ 0(pi)) - 1 J2 MKPi+i) ~ 0(Pi)) 

1<T 1>T 

= -*x>i(0(?j+i)-%))- 

;<fc 

Corollary El now implies that 

lEP(n)F(A(t;)) < -2A« - 7 ^ + 7 3 (n)F(A, 7 ) - t ^ m,(%+i) - %/)) + ft. 

We will use this bound for A as in the e-minimizer (k, m, q, A) and 7 = 0, i.e. 

±EV(n)F(A(v)) < -2Xu + V(n)F(X, 0) - t ^ - %)) + ft- 

The argument in Lemma El shows that |A| < A for a constant A that depends only on £, z/ 
and u and, hence, |72.| < K(en + c(AT)). Recalling the definition of ip{t) in ()6.12|) . in order 
to prove Proposition El it remains to show that 

V(n)F(X, 0) < 2X (m, q,X)-K= 2V(m)X k+1 - K, 

where we used (|3.4j) and where K > is a constant independent of t and t>. In fact, it is 
enough to show that 

V(n)F(X, 0) < 2X (m, q, A) = 2V(m)X k+1 , (A.59) 
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for all parameters t G [0, 1 — to] and v as in (|A.51|) because the functionals on both sides are 
continuous in these parameters and, even though the set defined in (jA.51|) is not a compact, 
the case of the end points will be proved in the following sections and (JA.59J) holds on 
the closure of (jA.51j) . Therefore, by continuity and compactness, strict inequality for each 
(t, v) will imply strict inequality uniformly over the entire set of parameters. The proof of 
flA.59|) repeats the proof of Proposition 5.7 in ^T] with only one modification that instead of 
log chx we consider flA.28|) and note that this function is also strictly convex in x because we 
eliminated the case when v is concentrated on one point in Section [Bl Instead of reproducing 
the proof in its entirety we will explain a very clear idea behind it by looking at a few cases. 
Let us first show that a nonstrict version of ()A.59j) . i.e. 

V(n)F(X, 0) < 2X (m, q, A) (A.60) 

always holds, even without the assumption (jA.51|) . (jA.25|) gives that F(X,0) = F 1 + F 2 
where 

F j = log / exp(aJ24 + Xa^duia). 
It is clear from the construction that for p £ I 



n p = ni = — for / < r g\ = g 2 p (A.61) 

and for p G J 

n p = n' l = — ioTl<a g\ = i]g 2 p . (A.62) 

In other words, n p is of the type m//2 whenever the corresponding random pair is fully 
correlated. Let us define a vector m = (m , . . . , m K ) by 

m p = 2n p for p as in (jA.61|) or (jA.62|) and m p = n p otherwise. (A. 63) 

A fact that plays a very important role below is that coordinates of m are not necessarily 
arranged in an increasing order. Let us first prove the following. 

Lemma 13 We have 

V(n)F(X, 0) = Vin^F 1 + F 2 ) < V{xa)F l + V(m)F 2 . (A.64) 

Proof. This follows by induction in (|3.2jl . For p such that m p = n p and gh g 2 are independent 
we have 

(F l + F 2 ) p = -\og^ p e* V n p {F l + F 2 ) p+1 <-\og^ p e*vn P {Fl +l + F 2 +l ) 
n p ti p 

1 I 

= — log E p exp mpi^+i + — log E p exp m p F p 2 +1 = F p x + F 2 . ( A.65) 
tn p tn p 

For p such that m p = 2n p and gl = ±g 2 we have 

1 2 tn 

(F 1 + F 2 ) p = -logEpexptipC^+FVi^— iogEpexp-^^a+i^+O 

Xlp ui p Z 

< — log E p exp m p Fl +1 + — log E p exp m p F p 2 +1 = F x y + F 2 (A.66) 
m p m p 
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where in the second line we used Holder's inequality. For p = this gives (|A.64|) . 

□ 

Lemma 14 Let m' be a nondecreasing permutation of the vector m. Then, 

V(m)F j < P(m')F j = V(m)X k+1 = X (m, q, A). (A.67) 

The first inequality means that the Parisi functional will decrease if m is not arranged 
in an increasing order. Lemma ITU follows from Lemma 5.12 in ^JJ which states the following. 
Given a function Q and numbers a > and m > 0, let 

T m a (Q)(x) = — logE exp mQ (x + g^/a) 
m 

where g is standard Gaussian. 

Lemma 15 (11 1$ ) If a, a' > and m > m! then for each x we have 

T m ,a ° T m , A ,(Q)(x) < T m t >a > o T m>a (Q)(x). (A.68) 

If a, a' > and m > m! then we can have equality only if Q is constant. 

Proof of Lemma 1141 The first inequality is obvious by Lemma ITB1 Equality in (jA.67j) 
follows by construction. The elements of m' are precisely the elements of m. The random 
variables g 3 p for p such that trip = mi are exactly 

\/T— tzj and Vtyf for I ^ a — 1 

and 

\/T— tzj and Vtyf, Vtyj +1 for I — a — 1. 

Obviously, 

Tm,a ° Tm,a' Tm,a+a' i (A. 69) 

which means that we can combine the random variables corresponding to the same value mi 
and since it is easy to check that in both cases the sum of these random variables is equal 
in distribution to zi defined in (jl,12j) . (jA.67|) follows. 

□ 

Combining Lemma ITS1 and fill we proved (jA.60|) . From the proof it is clear that there 
are only two places, (jA.66|) and (jA.68|) . where the inequality could become strict. It turns 
out that condition (jA.51|) ensures that gluing two sequences together occurs in such a way 
that at least in one of these two steps the inequality will become strict. We will not present 
the detailed proof here and refer a reader to Proposition 5.7 in ^Tj. We will explain the main 
idea by looking at several typical cases. 

Let us consider the case r > 2 in (jA.51|) . The case r = 1 is quite similar with the 
exception that the interval — q\ < v < q = was excluded in (jA.51|) because it requires a 
different approach and it will be postponed until the following sections. 
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For r > 2 and v as in (jA.51|) we have 

(a) |f| ^ [q r -i,Qr+i] or (b) \v\ E [q r -i,q r +i] and v = —\v\, i.e. 77 = — 1. 

Case (a). Let us assume for simplicity that q r -i < \v \ < g r _i since other cases are similar. 
This corresponds to the case a = r — 2 in (jA.53|) . Then we will split case (a) into two subcases: 

Qr~2 < \v\ < q r -i and m r _2 < m r _i/2; (A. 70) 

Qr-2 < M < Qr-i an d m r _i/2 < m r _ 2 - (A-71) 

Let us now see what happens when we combine the sequences at step 3 above. First of all, 
at step 1 the sequences n and (zl, z 2 ) will have subsequences 



m r _2/2 m r _2 m r _i m r 



(A.72) 



m r _ 2 /2 m r ^i/2 m r m r+ \ 

(yZ r -2i Z r —2) {z r —x : Z r -i) (z r , Z 2 ) {z^+i, ^r+l) 

At step 2 the sequences n' and (yp,Vp) will have subsequences 

(A.73) 

—vy r -v \y r iyr) \y r +iiyr+v ••• 

In both cases we write (z 1 , z 2 ) or (y 1 , y 2 ) whenever two coordinates are independent. When we 
glue these sequences together as step 3, the sequences n and (g*, g 2 ) will contain subsequences 

m r _ 2 /2 m r _ 2 /2 m r _ 2 m r _ x /2 m r _ x ... 

... (z r _ 2 ,z r _ 2 ) (y r _ 2 ,77y r _ 2 ) (i; 1 -!,^) (^-1,^-1) 0?,*; 2 ) ••• 1 j 

in the case (|A.70|) and 

m r _ 2 /2 m r _ 2 /2 m r _i/2 m r _ 2 m r _i ... ? - 

... (Z r _ 2 ,Z r _ 2 ) (y_2,r/y_ 2 ) (^-1,^-1) {YUX-x) {YrX) ••• 1 J 

in the case (|A.71|) . 

Suppose that (| A. 74(1 is the case. Then the strict inequality will appear when we apply 
equation (jA.66|) at the step when n p is equal to m r _i/2. Indeed, at this step 

Fi+i = Fi+i(- ■ ■ + z r -2 + y,- 2 + r r \ + z r _o, 



p+1 x p+1 



. . . + Z r _2 + Y r -2 + Y 2 _ 1 + Z r _j 



and y^x,^ 2 . 4 are independent and nondegenerate since E(y/_ 1 ) 2 — t(£'(q r -i) — £'(\v\)) > 
by (|A.70|) . Also, both functions x — > F^ +1 (x) are strictly convex because (jA.28|) is strictly 
convex and iteration in the Parisi functional ()3.2|) will preserve strict convexity. Therefore, 
Fp +1 and F 2 +1 are not collinear as functions of Z r _i with probability one over (Y^_ 1 ,Y 2 _ 1 ) 
and, therefore, Holder's inequality in (|A.66|) will be strict with probability one. 
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Now, suppose that ()A.75|) holds. Then after using Lemma ITBI V^vrijF 1 will be defined 
in terms of the sequences that contain subsequences 

. . . m r _ 2 m r _ 2 m r _i m r _ 2 m r _i ... , , 

7 V 7 V 1 V 1 V '°/ 

Z/ r _ 2 I r _2 r r-l r r 

In this case, m is not arranged in an increasing order, since m r _x > m r _ 2 , and Z^i,}^!.^ 
are nondegenerate. Therefore, when we rearrange these sequences in an increasing order by 
applying Lemma [TBT we will get strict inequality in (|A.68|) . 

Case (b). In this case the scenario of (jA.75|) can not occur and the fact that rj — — 1 
plays an important role. Suppose for certainty that g r _ x < \v\ < q r . ()A.72|) does not change 
but instead of (jA.73|) we will now have: 



m r _ 2 /2 m r _ 1( /2 m r _ x m r 

■ ■■ {y r -2,-y r -2) {y r -i,-y r -i) {vl^yl) {yl+nvl+i) ■■ 

When we glue this sequence with (|A.72|) we will get 

m r _ 2 /2 m r _ 2 /2 m r _i/2 m T ^i/2 m r _i 

... (Z r _ 2} Z r _ 2 ) (Y r _ 2 , —Y r _ 2 ) (Z r _i, Z r _i) (Y r _i,—Y r _i) (Y^-,Y r 



(A. 77) 



2^ 



(A. 78) 



The strict inequality will appear when we apply equation (jA.66|) at the step when n p is equal 
to m r _i/2 and 

F l+1 = F i + li ■ ■ + Z r-2 + Yr-2 + Z r _ x + Y r _ x ), 
Fp+1 = Fp+l(- ■ ■ + — Yr-2 + Zr-1 — Yr-l)- 

Random variables Y r _ 2 , Z r _ 2 are independent and nondegenerate and we can argue as in the 
case ()A.74j) above. All other cases in the proof of Proposition 5.6 in ^1] are very similar and 
|A~52|) holds. 



A. 4 Control of the close points. 

In Section IA.3I we obtained the control of the points v far from q r and in this section we 
will consider the remaining cases when qv-i < v < Qr+i or —q± < v < when r — 1. All 
arguments repeat the arguments of Section 5 in ^T] , so we will only consider the case when 

Qr-1 < v < q r . 

As in the previous section, let L 1; L 2 , . . . denote constants that depend only on z/, £ and u. 
Consider a function 

r(c) = inf \\£{y) - £{x) + (x - y)£{y)\ : < x, y < D, \x - y\ > c}. (A.79) 

Since £,"(x) > we have T(c) > for c > 0. In the notations of Theorem |U] the following 
holds. 
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Proposition 3 Suppose that g r _i < v < q r . If Lie 1 ^ < 1 — to then 

Hq r - v) < 1 - 1 ±-EV(n)F(A(v)) < 2^(t) - (1 ~ to) \ v - q r f + TZ, (A.80) 

iv Lj\ 

and if L 2 e 1 l 2 < (1 - t )T((l - to)/£i) &en 

- «) > 1 - to —KP{n)F{A{v)) < 2ip(t) + TZ. (A.81) 



Together with a similar result for q r < v < q r+ \ and the results of Section IA.31 this proves 
Theorem El We will again use Talagrand's interpolation for two copies. Given 

2 - - r, 

let us define sequences n and p in (jA.lj) and ()A.2jl by 

m mi m r _i 

= n = -7^1 n l = n r~l = -^—1 n r = m > n r+l = m r, • • • , tTfc+1 = m k 

and 

PO — <Z0> • • • j Pr-1 = Qr-lj Pr = U, Pr+1 = 9r> • • • j Pfc+2 = 9fc+l- 

Since p r = t>, we have r = r. Consider a sequence (l^, 1 , Yp 2 ) as in ()A.3|) . i.e. 

E(Yi) 2 = t(£(p p+1 )-a Pp )), 

Y 1 = Y p 2 for p < r and l^ 1 , Y p 2 are independent for p > r. 
Let (Zp, Z 2 ) be such that 

E(^) 2 = (i-t)(ew)-e'(p P )) 

for p < r — 1 and p > r, 

E(ZU) 2 = (l-t)(e(q r )-e(q r -i)) 

and Zj! =0. Let 

= Z 2 for p < r and Z 1 , Z 2 are independent for p > r. 

If we denote 

9p [ = Yi + Zi for < p < k + l,j = 1, 2 (A.82) 
then if follows from the construction that 

e (#p) 2 = C'(Pp+i) - C'(Pp) for p < r - l,p > r, 



E(^_x) 2 = (e'(9r) - rfor-l)) - t(^(?r) " E(^') 2 = t(£'(g r ) - £») (A.83) 
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and 

9p = for p < r and g^g 2 - independent for p > r. 
If we define a point a e [v, q r ] by 

C(a) = t?(v) + (1 - t)e(q r ) (A.84) 

then ifPSjl can be rewritten as 
If we define a new sequence p' by 

A) — 90, • • • > Pr-l = Qr-l,Pr = a > Pr+l = Qr, ■ ■ ■ , Pk+2 = Qk+1 

obtained by inserting a into the sequence q, we finally get 

naif = C'H+i) - £' W for all p < fc + 1 (A.85) 

and 

g p = g 2 p for p < r and g p are independent for p > r. (A. 86) 

Plugging the definition (jA.82|) into (jA.25|) we get 

F(A, 7 ) = log/ exp^o-j- ^ ^ + ^A(a J ) 2 + 7( x 1 a 2 )rfz/( ( T 1 )rfz/( ( T 2 ). (A.87) 

i<2 p<fc+l j<2 

Let us define 

7(7, m, u) = P(n)F(A, 7) (A.88) 

where we made the dependence of the right hand side on the parameters (7, m, v) explicit. 
In order to apply Corollary let us first note that from the construction of sequences n and 
p we have 



-2t ^ M^Pi+i) - B{pi)) ~ t MHPi+i) ~ 0(Pi)) 

1<T 1>T 

-t J2 rni(8(Qi+i) ~ B(H)) - tm r ^(9(v) - % r _ x )) 

Kr-2 

-tm(6(q r ) - 9(v)) - t mi(9(qi+i) - %)) 

r<l<k 

-t^2mi{9(qi +1 ) - 6(qi)) - t(m - m r _!)(% r ) -9{v)). 



Kk 



Corollary El and (|A.58|) now imply that 

—KP{n)F{A{v)) < -2Aw- 7 f + V(^,m,v) (A.89) 



-t^2 mi {9{q l+1 ) - %,)) - t(m - m r ^){9{q r ) - 9{v)) + K. 



Kk 
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One can easily check using the argument of Lemma ^ that 

V(0, m, v) = 2T(m, £'(a)) (A.90) 
where T was defined in fjA~29j) . (lA~33l) implies that 

V(0, m r _i, f ) = 2X (m, qr, A) 
and, therefore, (|A.89|) with 7 = 0,m = m r _i implies that 

— KP(n)F(A(v)) < 2ij){t) + K. 

In order to prove Proposition El we will perturb parameters m and 7 around these values 
0, m r _i and use the properties of e-minimizer from the previous section. The fundamental 
connection of the bound (jA.89|) to the properties of e-minimizer lies in the following fact: 

dV 

_(7, mr _ 1 , U )| 7=0 = -tf{?{a)). (A.91) 

The proof follows from straightforward computation and is given in Lemma 5.8 in jTT]. Also, 
similar to (|A.45|) . we have 

d 2 V 



< L. (A.92) 



cfy 2 

We are now ready to prove Proposition El 

Proof of Proposition |31 Let us consider a function 

a(j) = V(j, m r _i, v) - jv, 

which is the part of the bound (jA.89|) for m = m r _i that depends on 7. By (jA.91|) we have 

h(v) := a'(0) = -£/'(£'(a)) - v = -U'(t?(v) + (1 - t)i'{q T )) - v, 

since a was defined in (jA.84j) . By ()A.41|) we have h(q r ) = and 

ti{q r ) = -te(q r )U"(e(q r )) - 1. (A.93) 

Using (TOTl) . 

h'(q r ) = -t"{q r )U"{£{q r )) - 1 + (1 - t)t" {q r )U" {t,{q r )) 

< ^e 1/6 + (l-t)e(q r )U"(a<lr)). (A.94) 



We will now show that 



Le l/Q < 1 - 1 h\q r ) < -^-A (A.95) 



If (q r )U"(£' {q r )) < 1/2 then ijOSj) gives 



h\q r ) <--!<--. 
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If -£"(qr)U"(£'(q r )) > 1/2 then 

1 



by (|A.45jl . and (jA.94jl gives 



< -2[/"(£'(g r )) < L 



1 - t . 1 - t 



ti(q r )<Le 1 " i - — < { 

where the last inequality holds if ALe 1 ^ < 1 — to- This proves ()A.95|) . Since (jA.45|) implies 
that < L and since h(q r ) = 0, we can write 

h(v) >(v- q r )h'(q r ) - L(v - q r ) 2 > -(q r - v){l - t ) 

if q r - v < (1 - t )/8L and if (fA~95jl holds. (fA~92jl implies that |a"(7)| < L and we finally 

get 

(y'(0) 2 h(v) 2 
inf «( 7 ) < inf (a(0) + a'(0) 7 + L 7 2 ) < a(0) - — ^- = a(0) - 

< 2X o (m,q,A)-i(l-t o ) 2 (^-gr0 2 . (A.96) 

Applying this to the bound (JA.89J) proves (|A.80J) . Note that the infimum was achieved on 
7 = — q'(0)/2L and that (|A.45|) implies that | 7 | < L. As we explained in the remark following 
Corollary El the bound (|A.27J) is used only for | 7 | < L. 

Next, we will prove ()A.81|) . If —h(v) = U'(£'(a)) + v ^ then we can simply use the 
first inequality in (|A.96[) . Let us assume now that U'(£'(a)) = —v. Let us set 7 = in the 
bound (|A.89|) and consider the derivative of this bound in m at m = m r _i, i.e. 

= U(£'(a)) - t(6(q r ) - 9{v)) by (lA~90l) and (lA~35l) 

= U(te(v) + (l-t)?(q r ))-t(0(q r )-9(v)). by ®M 

Since we assumed that U'(£'(a)) = —v, 

= -(e(v)-Z'(qr))v-(9(q r )-e(v)) 

= £(<Zr) - + - 9r)e'(?r) < -r(<Zr ~ «), 

where the last inequality follows from the definition (|A.79j) . By ()A.42|) . D(t) is concave in t 
and, therefore, 

D(l) < D(t) + (1 - £)£>'(*) < D(t) - (1 - t) 7 ( 9r - u). 

By (|A~46l) . 

D(l) = [/(£'(<;)) - (% r ) - *(«)) = 2/(«) > -^ V2 - 
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We get 

D(t) > (1 - t)j{q r -v)- Le 1/2 > 0, 

if (1 — to)^{q r — v) > Le 1 / 2 , which is true under the conditions in (JA.81j) and one can finish 
the proof as in ()A.96|) . 



B Cases reducible to the classical SK model. 

We will now show that only the case of d < u < D in Theorem ^ is different from the classical 
SK model. First of all, d = D means that S = {—yd, +va} which is precisely the case of 
the SK model. If measure v has nonzero mass at both points ±^/d then z/(cr) is proportional 
to exp ha for some external field parameter /i 6 1. Otherwise, if v is concentrated at one 
point, the statement of Theorem ^becomes trivial. 

It remains to consider the cases of d < D and u = d or u = D. We will only consider 
the case u = d, since the case u = D is similar. Let us consider a set 



and a function 



Since, by (jl.lj) . 



U N (e) = {a : R 1A e [d,d + e}} 
F N (e) = ^Elog / expH N (a-)du((r) 

7V JU N (e) 



E 



/ exp H]y((T)dl>(cT) < oo, 

Jt, n 



the function expifjv(cr) is ^-integrable with respect to a almost surely and, therefore, by 
the monotone convergence theorem, 

lim / exp H]y(cr)dv((T) = / exp H]y(a)di'(cr) a.s. 

e ^°Ju N (e) ' J{Ri,i=d} 

Using the monotone convergence theorem once again implies 



\imF N (e) = P N := ^Elog / exp H N (cr)dv(<r) . (B.l) 

'{Ri,i=d} 



If we choose a sequence (en) so that \Fn(sn) ~ Pn\ < N l , in order to prove Theorem [T] for 
u — d, it is enough to show that 

lim P N = V(£,d). (B.2) 

We will prove this by considering two separate cases. 

Case 1. v{\o 2 = d}) = 0. This means that the measure v has no atoms at the points 
±Vd and, therefore, v({R\^ = d}) = and Pn = —oo. To prove ()B.2j) . we need to show 
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that V(£, d) = — oo. Let us, for example, take k — 1 and q\ = 0. For this choice of m, g we 
have 

X (m, q, A) = logE ^ exp^azA/^n) + Ao- 2 jdz/(a). 

= log y exp(~o- 2 £'(w) + Aa 2 )rfz/(a). 



Therefore, 



-Xd + X (m, q, A) = log ^ exp(i<7 2 £'0) + A(a 2 - d) )di/(er) 
and, by the monotone convergence theorem, 

lim (— Xd + X (m, q, A)) = log / expf-<7 2 £ ; '(u))dv(a) = — oo, 

A ^-°° J{ ( T 2 =d} V2 ' 

since we assumed that v{\o 2 = d}) = 0. Clearly, (J1.16)) yields that V(£, d) = —oo. 

Case 2. ^({cr 2 = d}) > 0. This means that measure v has at least one atom at the 
points ±y/d. Consider a probability measure v defined by 

Condition (jl.7|) implies that R^i = d if and only if of = d for all % < N. In other words, 

{Pi,i = d} = S^, where S d = {— yd, Vd}. 
With these notations, Pat in (|B.1|) can be written as 

P w = logz/({a 2 = 4) + P^ (B.4) 

where 

N 



P/v = -j-Elog / exp H N {er)dv{cr). 



But Pv exactly falls into the case d = D considered above because v({& 2 = d}) = 1, and, 
therefore, its limit can be written as follows. If we consider 



Y k+1 = log / exp^a ^ z^du(a 



0<p<k 



define Y\ recursively as in 1)1.14)1 and let 



then 



2 

KKk 



lim P N = miV k (m,q), 
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where the infimum is over all choices of k, m and q, which by (jB.4|) implies 

lim P N = logi/({<j 2 = d}) + in£V k (m,q). 

N^oo 

Equivalently, this can be written as follows. If we consider 

Yfc+i = log v{{a 2 = d}) + log / exp(cr ^ z p ^di?(a), (B.5) 

0<p<k 

define Yi recursively as in ()1.14|) and let 

V k (m,q)=Y -l Yl m{e{qi + i)-e{ qi )) (B.6) 



2 

KKk 



lim P N = MV k (m,q). (B.7) 



then 



By definition ()B.3j) of measure u, Y k+ i in (|B.5j) can be also written as 

Y k+1 = log / expfo- y~] z p )dv(a). (B.8) 
J W 2 =d} v < p < fe ' 

In order to prove ()B.2j) . we will show that V(£,,d) is equal to the right hand side of ()B.7j) . 
The definition of V(£, d) given by ()1.13|) - f)l . 16|) can be written equivalently as follows. If 
we consider 

X k+l = log / exp(a z p + A ( ff2 - d))M<r), ( B - 9 ) 

0<p<fc 

define X\ recursively as in (jl.l4j) . and define 

V k (m,q,X,d) = X -^ Yl m{e{qi + i)-e{ qi )), (B.10) 

l<l<k 

then 

V(^d) = MV k (m,q,X,d), (B.ll) 

where the infimum is taken over all A, k, m and q. Since d < a 2 for a G S, X^ +1 in (jB.9|) 
is increasing in A which implies that X is also increasing in A. Therefore, for any fixed m 
and q to minimize the right hand side of (jB.ll|) over A one should let A — > — oo. By the 
monotone convergence theorem, almost surely, 

lim X k+1 = / expfcr z p )du(a) = Y k+x . 

J {a -d]f 0<p<k 

Using the monotone convergence theorem repeatedly in the recursive construction (|1.14jl 
gives liniA^-oo X = Y and comparing (jB.6|) and (|B.10|) we get 

inf P k (m, q,X,d) = V k (m,q). 

A 

Combining this with (|B~7) and (IBTTJ) gives (|B~2l) . 
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